[인공지능개론] Regression④

앞서 Linear Regression을 PyTorch로 구현했던 것처럼, Logistic Regression과 Softmax Regression도 PyTorch로 구현해보자. 이것도 직접 작성할 수도 있고 torch.nn 모듈을 활용하여 쉽게 구현할 수도 있다.

Logistic Regression

직접 작성한 코드를 먼저 확인해보자.

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

x_data = [[1,2],
          [2,3],
          [3,1],
          [4,3],
          [5,3],
          [6,2]]   //input dimension = 2
y_data = [[0],
          [0],
          [0],
          [1],
          [1],   // output dimension = 1
          [1]]   // 0/1 binary classification

x_train = torch.FloatTensor(x_data)
y_train = torch.FloatTensor(y_data)

W = torch.zeros((2,1), requires_grad = True)
b = torch.zeros(1, requires_grad = True)

optimizer = optim.SGD([W,b], lr = 1)   //stochastic gradient descent

nb_epochs = 1000
for epoch in range(nb_epochs + 1):

  hypothesis = 1 / (1 + torch.exp(-(x_train.matmul(W) + b)))   // logistic hypothesis
  cost = -(y_train * torch.log(hypothesis) + (1-y_train) * torch.log(1-hypothesis)).mean()
	// cost function from BCE(binary cross entropy)
    
  optimizer.zero_grad()   // should initilaize gradients for new epoch
  cost.backward()
  optimizer.step()

  if epoch % 100 == 0:
    print('Epoch {:4d}/{} Cost: {:.6f}'.format(epoch, nb_epochs, cost.item()))

if epoch == nb_epochs:
  prediction = hypothesis >= torch.FloatTensor([0.5])
  print(prediction)

 

 

nn.Module를 이용하여 코드를 바꿔보자.

 

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

class BinaryClassifier(nn.Module):
  def __init__(self):
    super().__init__()
    self.model = nn.Sequential(
    	nn.Linear(2,1),			# Wx+b
    	nn.Sigmoid()			# sigmoid(Wx+b)
    )

  def forward(self,x):
    return self.model(x)		# model(x) = sigmoid(Wx+b)
    
x_data = [[1,2],
          [2,3],
          [3,1],
          [4,3],
          [5,3],
          [6,2]]
y_data = [[0],
          [0],
          [0],
          [1],
          [1],
          [1]]

x_train = torch.FloatTensor(x_data)
y_train = torch.FloatTensor(y_data)

model = BinaryClassifier()

optimizer = optim.SGD(model.parameters(), lr = 1)	# stochastic gradient descent

nb_epochs = 1000
for epoch in range(nb_epochs+1):
  hypothesis = model(x_train)
  cost = F.binary_cross_entropy(hypothesis, y_train)	# BCE
  optimizer.zero_grad()
  cost.backward()
  optimizer.step()

  if epoch % 100 == 0:
    prediction = hypothesis >= torch.FloatTensor([0.5])
    correct_prediction = prediction.float() == y_train
    accuracy = correct_prediction.sum().item() / len(correct_prediction)
    print('Epoch {:4d}/{} Cost: {:6f} Accuracy {:2.2f}%'.format(epoch,nb_epochs,cost.item(),accuracy*100))

Logistic Regression을 이용한 가설 모델을 세우면 BinaryClassifier로 sigmoid(Wx+b)를 정의하고 model = BinaryClassifier() 으로 나타낼 수 있다. 

BCE를 이용한 cost function은 cost = torch.nn.functional.binary_cross_entropy(hypothesis, y_train)로 나타낼 수 있다.

Stochastic Gradient Descent은 optimizer = torch.optim.SGD(model.parameters(), lr =$\alpha$)로 나타낼 수 있다.

 

Softmax Regression

직접 작성한 코드를 먼저 확인해보자.

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)
x_train = [[1,2,1,1],
           [2,1,3,2],
           [3,1,3,4],
           [4,1,5,5],
           [1,7,5,5],
           [1,2,5,6],
           [1,6,6,6],
           [1,7,7,7]]
y_train = [2,2,2,1,1,1,0,0]			# class 3개
x_train = torch.FloatTensor(x_train)
y_train = torch.LongTensor(y_train)

y_one_hot = torch.zeros(8,3)
y_one_hot.scatter_(1,y_train.unsqueeze(1),1)

W = torch.zeros((4,3), requires_grad = True)
b = torch.zeros(1, requires_grad = True)

optimizer = optim.SGD([W,b], lr=0.1)

nb_epochs = 1000
for epoch in range(nb_epochs + 1):
  # Using Cross Entropy as Cost Function
  z = x_train.matmul(W) + b			
  cost = F.cross_entropy(z, y_train)
  
  # Using Multiple Loss as Cost Function
  # hypothesis = F.softmax(x_train.matmul(W) + b, dim=1)
  # cost = (y_one_hot * -torch.log(hypothesis)).sum(dim=1).mean()
  

  optimizer.zero_grad()
  cost.backward()
  optimizer.step()

  if epoch % 100 == 0:
    print('Epoch {:4d}/{} Cost: {:.6f}'.format(
        epoch, nb_epochs, cost.item()
    ))

nn.Module를 이용하여 코드를 바꿔보자.

torch에서 제공하는 cross_entropy 에는 softmax가 포함되어있으므로 별도의 작업이 필요없다.

 

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

class SoftmaxClassifierModel(nn.Module):
  def __init__(self):
    super().__init__()
    self.linear = nn.Linear(4,3)

  def forward(self,x):
    return self.linear(x)

x_train = [[1,2,1,1],
           [2,1,3,2],
           [3,1,3,4],
           [4,1,5,5],
           [1,7,5,5],
           [1,2,5,6],
           [1,6,6,6],
           [1,7,7,7]]
y_train = [2,2,2,1,1,1,0,0]
x_train = torch.FloatTensor(x_train)
y_train = torch.LongTensor(y_train)

y_one_hot = torch.zeros(8,3)
y_one_hot.scatter_(1,y_train.unsqueeze(1),1)

model = SoftmaxClassifierModel()
optimizer = optim.SGD(model.parameters(), lr = 0.1)

nb_epochs = 1000
for epoch in range(nb_epochs + 1):

  prediction = model(x_train)
  cost = F.cross_entropy(prediction, y_train)

  optimizer.zero_grad()
  cost.backward()
  optimizer.step()

  if epoch % 100 == 0:
    print('Epoch {:4d}/{} Cost: {:.6f}'.format(epoch, nb_epochs, cost.item()))

Softmax Regression을 이용한 가설 모델을 세우면 SoftmaxClassifierModel로 $\overline{Y}=Wx+b$를 정의하고 model = SoftmaxClassifierModel () 으로 나타낼 수 있다. 

Cross Entropy를 이용한 cost function은 cost = torch.nn.functional.cross_entropy(prediction, y_train)로 나타낼 수 있다.

Stochastic Gradient Descent은 optimizer = torch.optim.SGD(model.parameters(), lr =$\alpha$)로 나타낼 수 있다.

[출처] 한양대학교 장준혁 교수님 인공지능개론 수업