Video annotation library

Hello everyone,
For my PhD, I’m building an accelerometer data decoder using covnets. But first I need to annotate videos of animal behaviour, and I’d like to have some software or python code/library that allows me to predefine the classes of behaviours to annotate, assign them to numbers on my keyboard, and then just press those buttons as I watch the video to annotate it. And ideally that would yield a csv with the times of when each behaviour happened. There’s nothing on google that I could find that fits this description.

Any suggestiones are greatly appreciated!


Hello, Francis @franfram , welcome to the Python world!

There are a lot of options you could choose from to do with whether to make this software yourself or use software someone else has made, and how elaborate to make it. I won’t attempt to get into that.

One thing I like to do in a situation like this is to imagine, what is the simplest Python program I could possibly write which would be helpful for this. And here is one Python program which is simple, and could be helpful. When you run the program from the command line, it does a loop: it waits for you to type anything followed by “enter”. As soon as you do so, it writes that text, and the current time, as a row appended to a CSV file. Then it loops back to waiting for you to type a key. When you type control-C, Python interrupts the program. Then you open the CSV file with a spreadsheet application, and analyse. You will need to record what each key-press means in your analysis somewhere else. You will need to identify each video, and time when you started it, somewhere else.

Python has a csv module which makes the CSV formatting easy. You will want to read up on files. You will want to read all of the Tutorial, especially the part on reading and writing files. The builtin function input() will give you whatever you typed, and requires the “Enter” keypress to deliver it.

I hope this sketch gives you some ideas, and some areas to start investigating. Good luck with your research!
—Jim DeLaHunt, Vancouver, Canada

For better accuracy you can improve the program to control a video player and/or even get the actual played time position from it.

For example VLC player can be controlled like that.

Or if you want to invest a lot of work into the program - you can implement your own GUI. There is a Python library which allows you to use VLC as a component: python-vlc · PyPI

Hi, Jim.
Thank you for you time.
The python program you suggest is great. The only issue I have is with the time, since I don’t have to record the current time but the video’s time. Do you have any idea on how would you the same thing with extracting and writing a videos time?


Hi Vláclav!
Thank you for answering.
The control of the VLC player seems like a great idea!


Bear in mind, I am suggesting ideas for the simplest possible program which can do the job.

Every time you start a video, also make an entry to the time-recording program. This entry will correspond to approximately 0:00 time code in the video, but will have few seconds error. Every entry you make about that video will have the same amount of error. Once data collection is done, go back and replay the video around one of the events. Determine the exact time code of the event. Determine the error in the timestamp of the recorded entry relative to the exact time code. Correct this error for every event related to that video. You now have a list of video timestamps in terms of video timecode.

1 Like

Thats great, Jim. Thank you again