Ia tshark and subprocess

Hello all !

I’m new to python and working on data streaming for anomaly detection.
As input data i use tshark to capture network paquets, send them to a kafka queue, to train neural networks to detect anomalies. I already have anomaly detection model that works on static data. I’m stuck on the data streaming ingestion stage.
I use tshark to capture paquets in json format. So this code works

json_str = sp.check_output(“tshark -c 5 -T json -e ip.src -e ip.dst -e ip.flags.df -e ip.flags.mf -e ip.flags.rb -e ip.len -e ip.ttl”.split(’ ‘)).decode(‘utf-8’)
tshark_pkts = json.loads(json_str)
data = [pkt[’_source’][‘layers’] for pkt in tshark_pkts]


producer.send(topic, data)

but i have to wait for tshark process to stop before the data are sent to kafka topic.
I’m wondering how can i send the data in real-time to kafka topic.

thanks for your help