Unable to read from file into neural network

I have come up with the following code for reading some data from a file into a neural network. Input 1 is a csv with 2 columns (two input variables) and input 2 single column y values for the function. Finally I need to do some prediction (that part of the code is not yet written because this itself is not working but if someone can help it would be good). For that another file 3 will be read with 2 variables input and a fourth file would be written out containing computed values of y.

However the first part of the code is not working. Please help. Please keep code as simple as possible, with minimum changes to this script.

Import torch
import torch.nn as nn
import tensorflow as tf
import numpy as np

data_x = torch.from_numpy(
np.genfromtxt(“C:/NewHS/DataScience/comp/tensor1.csv”, dtype=‘float64’, delimiter=“,”)
)
data_y = torch.from_numpy(
np.genfromtxt(“C:/NewHS/DataScience/comp/tensor2.csv”, dtype=‘float64’, delimiter=“,”)
)

n_input, n_hidden, n_out, batch_size, learning_rate = 2, 15, 1, 100, 0.01

model=nn.Sequential(nn.Linear(n_input, n_hidden), nn.ReLU(), nn.Linear(n_hidden, n_out), nn.Sigmoid())

print(model)

loss_function = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

losses =
for epoch in range(5000):
pred_y = model(data_x)
loss = loss_function(pred_y, data_y)
losses.append(loss.item())
model.zero_grad()
loss.backward()
optimizer.step()

import matplotlib.pyplot as plt
plt.plot(losses)
plt.ylabel(‘loss’)
plt.xlabel(‘epoch’)
plt.title(“Learning rate %f”%(learning_rate))
plt.show()

What is the error you get?

It prints the model fine where the print command is. A little after that says:

RuntimeError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_10908\440809175.py in <cell line: 29>()
28 losses =
29 for epoch in range(5000):
—> 30 pred_y = model(data_x)
31 loss = loss_function(pred_y, data_y)
32 losses.append(loss.item())

D:\WinPython\python-3.10.5.amd64\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1193 or _global_forward_hooks or _global_forward_pre_hooks):
→ 1194 return forward_call(*input, **kwargs)
1195 # Do not call functions when jit is used
1196 full_backward_hooks, non_full_backward_hooks = ,

D:\WinPython\python-3.10.5.amd64\lib\site-packages\torch\nn\modules\container.py in forward(self, input)
202 def forward(self, input):
203 for module in self:
→ 204 input = module(input)
205 return input
206

D:\WinPython\python-3.10.5.amd64\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1193 or _global_forward_hooks or _global_forward_pre_hooks):
→ 1194 return forward_call(*input, **kwargs)
1195 # Do not call functions when jit is used
1196 full_backward_hooks, non_full_backward_hooks = ,

D:\WinPython\python-3.10.5.amd64\lib\site-packages\torch\nn\modules\linear.py in forward(self, input)
112
113 def forward(self, input: Tensor) → Tensor:
→ 114 return F.linear(input, self.weight, self.bias)
115
116 def extra_repr(self) → str:

RuntimeError: mat1 and mat2 must have the same dtype

Try with dtype float32? That’s the standard dtype in pytorch.

This is generally much more of a pytorch question. There is a forum for that: https://discuss.pytorch.org/

Does seem to work!!! It has moved forward, though learning function doesnt seem to be converging. Could be data issue now.

To give an idea of the new issue, I created one file with random values of x1 and x2. The second file is y, with values calculated using the function x10.1 + x20.9 + x1x2log(x1).

Now I need to see how well the Neural Network is able to mimic the function when prediction. Maybe my Model is wrong. Could you please suggest a better model?

There is one more issue. I am getting an intermittent message that tensor should be [100,1] and is [100]. Couldnt record it, but it was a UserWarning that doesnt appear most of the time. Maybe that is the issue why convergence is not occuring. Any hints? It could be referring to data_y variable which is [100] and not [100,1] (previous to the file reading, I was randomly generating data which was [100,1].

Seems to be working now, updated code below - except for the prediction. Whatever input parameters I give, output seems to be fixed at 9481.xxx

Modelling:
torch.tensor([10.0,25.0]
torch.tensor([1.0,1.0]

gives same answer. any clues?

This question is more or less resolved. I switched to Adam optimiser. That helped. This was mostly trial and error though. I seem to need only 5 hidden layers, adding more doesnt seem to help. Also going from 5000 iterations to 10000 helps marginally. Decreasing batch size doesnt help, keeping batch_size = input size does. I wonder why, when all websites suggest decreasing batch size. Any rational explanation behind these trial and error results would be appreciated.