Need help figuring this out, trying to make a http server using a socket and the standard library as much as possible

Python Help
1

hello thanks for reading this and helping out, i really appreciate it.

this is only my second project after learning the basics of python/programming. i am trying to make a really basic http server for my own use. it is only going to support GET and POST. i am trying to use the standard library as much as possible just because. and i have imported a few modules but those are there just in case pretty much.

this code is a bit messy. there is a thing or two there that doesnt need to be there, just there because i am trying to figure things out.

my problem here is reading the recieved data. i have not got to the part where i send the client some data yet, because i cannot get this program to tell if the path is in the recieved data or not, i was trying to use a for loop through data3 if client_choice in data3 then send a response. and first i just wanted to be sure that i could read the path: client_choice or not. and i have had absolutely no luck. so thats where im at with this, i deleted the for loop and tried something else and have not changed it back.

any help is really appreciated, thank you.

this is a edit: i just want to say again that this is not complete and the way it is because i am just trying to figure it out bit by bit, thanks for helping

‘’’
import os
import socket
import io
import sys
import requests
import csv
import time
import re
Path = list()
list_of_path = list()
client_choice = str(“”)
download_path = list()
#download_path.join(Path)
garbage = list()
def get_path():
path = input(“enter absolute file path to serve from, and to save POST’ed files in: “)
while not os.path.isdir(path):
print(“path not accepted, try again”)
path = input(“enter absolute file path to serve from, and to save POST’ed files in: “)
if os.path.isdir(path):
print(“path accepted”)
for item in os.listdir(path):
list_of_path.append(item)
Path.append(path)
client_choice = path
def make_server():
HOST = “192.168.1.233”
PORT = 8025
try:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM,) as server1: # this server does not work, it cant read recieved data correctly or at all idk
server1.bind((HOST, PORT))
server1.listen()
print(f"server is working on IP: {HOST} and PORT: {PORT}”)
conn, addr = server1.accept()
print(f"new connection from: {addr}”)
data = conn.recv(4096)
data1 = data.decode()
data3 = data1.split(” “)
data2 = [data1[i:i+50] for i in range(0, len(data1), 50)]
print(data2)
print(data3)
if client_choice in data3: #any(client_choice in x for x in data3) # for loop was here
x = data1.find(client_choice) # changed line.find(client_choice) to data1.find
y = str(””)
for letter in data1[x:]:
y += letter
if letter == " “:
download_path.append(y)
conn.send(b"HTTP/1.1 200\r\n\r\nthis is what you see if it read client_choice”)
#elif “GET” in data3:
#conn.send(b"HTTP/1.1 200\r\n\r\nthis is reply to get")
else:
conn.send(b"HTTP/1.1 200\r\n\r\nthis is reply to NO get and NO client choice")
except socket.error:
time.sleep(5)
finally:
make_server()

if name == “main”:
get_path()
make_server()
‘’’

2

I take it that you specifically DON’T want to use modules like http.server (also part of the standard library)?

3

yes that is correct. i do not want to use http.server. this is just for my own use with curl mostly in my own LAN or WLAN. and i thought it would be good to do it that way to figure things out

4

Then, given that your goal is to actually understand how HTTP works, I would strongly recommend reading the relevant RFCs (eg RFC 2616) and stepping through each part of what’s going on. You’ll need to read from the socket and keep it in memory until you can process the full set of request headers; there are no guarantees with conn.recv() about how much it will give you, so you might need to query more than once.

It may actually be easier for you to read through the source code for http.server than to start from nothing.

1 Like
5

this is only my second project after learning the basics of
python/programming. i am trying to make a really basic http server for
my own use. it is only going to support GET and POST. i am trying to
use the standard library as much as possible just because. and i have
imported a few modules but those are there just in case pretty much.

Using socket directly is pretty low level, but good for figuring out
the core of what’s going on.

I’m going to put some remark inline in the code below.

A couple of high level things about make_server:

You really want a while-loop to process requests, calling the function
again is recursive and not how we repeat things. So change:

 def make_server():
     ... do one request ...
     make_server()

into:

 def make_server():
     ... set up the socket ...
     while True:
         ... accept a connect and handle the request ...

When you rnake a server you create the socket once, and then run a loop
accepting connections and processing each connection.

Put a lot of pint() statements in the code to see where you are and
what you’re getting.

my problem here is reading the recieved data. i have not got to the
part where i send the client some data yet, because i cannot get this
program to tell if the path is in the recieved data or not, i was
trying to use a for loop through data3 if client_choice in data3 then
send a response. and first i just wanted to be sure that i could read
the path: client_choice or not. and i have had absolutely no luck. so
thats where im at with this, i deleted the for loop and tried something
else and have not changed it back.

Do you know the basic structure of an HTTP request? Have you print()ed
out the decoded data? You should be seeing something like:

 GET /sub/path HTTP/1.1
 Host: 192.168.1.233

A POST will be more elaborate.

What are you using to test a request? wget, curl, etc?

Someone will complain about all these global variables. They’re right,
but they’re not the root of your problems.

 Path = list([])

Please call this “paths”. Using a singlular name for a list is a bit
jarring.

Also, you can just make an empty list as []. You don’t need
list([]), which makes an empty list from an empty list.

 list_of_path = list([])
 client_choice = str("")

Same here: "" is an empty string. str("") makes an empty string from
an empty string.

 def get_path():
     path = input("enter absolute file path to serve from, and to save POST'ed files in: ")
     while not os.path.isdir(path):
         print("path not accepted, try again")
         path = input("enter absolute file path to serve from, and to save POST'ed files in: ")
     if os.path.isdir(path):
         print("path accepted")

You don;'t technically need the if test here: you know that
os.path.isdir(path) is true because you exiting the while-loop.

 def make_server():
     HOST = "192.168.1.233"
     PORT = 8025

There two settings values probably should be globals. But again, not a
cause of problems.

 try:

Get rid of the try/except, it’s obscrubing exceptions and their
sources. Let them out! That way you get a nice traceback of exactly
where they were raised and the exception itself. In normal operation
this code shouldn’t cause exceptions.

 with socket.socket(socket.AF_INET, socket.SOCK_STREAM,) as server1:  # this server does not work, it cant read recieved data correctly or at all idk
      server1.bind((HOST, PORT))
      server1.listen()
      print(f"server is working on IP: {HOST} and PORT: {PORT}")

The code below this point should be in a while True: loop. You’ve made
the socket and listened. Now you loop, accepting connections and
processing each connection.

  conn, addr = server1.accept()
  print(f"new connection from: {addr}")
  data = conn.recv(4096)
  data1 = data.decode()
  data3 = data1.split(" ")
  data2 = [data1[i:i+50] for i in range(0, len(data1), 50)]
  print(data2)
  print(data3)

Put finer grained print()s in here (above). Print data1 and so on as
each value is made. See the data as they are transformed.

A more normal approach would probably be to wrap the connection in a
text file, and read lines of text.

An HTTP request normally has an opening line of the form: method
path HTTP/version, like GET /something HTTP/1.1. You should be grabbing the opening line and
examining that. Try splitting data1 on "\n" (a newline). That should
get you lines of text. Then examine the first line.

An HTTP response takes the form of a leading line: code info and
then following lines with further response data, if any. So your
responses should be line b"200 remark here\r\n" with no leading
HTTP/1.1 text.

 except socket.error:
     time.sleep(5)

That catches and conceals any socket error. At the very least you
should:

 except socket.error as e:
     print("GOT SOCKET ERROR!", e)

to show you the error. But in normal use you shouldn’t expect these
anyway, so probably best to drop the try/except entirely and let them
out.