# Help with malicious repository **URL:** \ **Category:** Python Help\ **Tags:** help\ **Created:** [July 25, 2022, 12:44pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677 "2022-07-25T12:44:56Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![farhaan710](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/farhaan710/32/8251_2.png) [@farhaan710](https://discuss.python.org/u/farhaan710)\ **Post date:** [July 25, 2022, 12:44pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/1 "2022-07-25T12:44:56Z")
I need to create a project in python which checks the maliciousness of a repository. ---
**Author:** ![ndc86430](https://avatars.discourse-cdn.com/v4/letter/n/d6d6ee/32.png) [@ndc86430](https://discuss.python.org/u/ndc86430)\ **Post date:** [July 25, 2022, 5:53pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/2 "2022-07-25T17:53:33Z")
Define “maliciousness”. ---
**Author:** ![farhaan710](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/farhaan710/32/8251_2.png) [@farhaan710](https://discuss.python.org/u/farhaan710)\ **Post date:** [July 25, 2022, 7:45pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/3 "2022-07-25T19:45:03Z")
I need to check the repository if it contains any harmful code. It is malicious if it contains malicious code. ---
**Author:** ![vbrozik](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vbrozik/32/7423_2.png) [@vbrozik](https://discuss.python.org/u/vbrozik)\ **Post date:** [July 25, 2022, 8:22pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/4 "2022-07-25T20:22:28Z")
> [@farhaan710](#): > > check the repository if it contains any harmful code This is a very complex task. What is your experience? 1. There is no simple and clear definition of a harmful code. It even differs based on your requirements. 2. Even if you resolve 1. then by definition it is impossible to detect “any harmful code” because this is a very quickly moving target. (always new platforms, languages, libraries, APIs, vulnerabilities, obfuscation techniques…) 3. Because of 2. antimalware programs use many techniques including: pattern matching, heuristics, AI, sandboxing and behaviour analysis. Based on your question I am guessing you are not going to develop your own antimalware. Then your first step would be to check the existing solutions for malware and vulnerability detection in source code. Unfortunately I think that almost none of the solutions aims at detection of intentional maliciousness. They are rather intended for detecting vulnerabilities caused by programmers negligence, lack of knowledge and use of vulnerable components. Here are interesting lists I was able to find: - [Source Code Security Analyzers](https://www.nist.gov/itl/ssd/software-quality-group/source-code-security-analyzers) - [Source Code Analysis Tools | OWASP Foundation](https://owasp.org/www-community/Source_Code_Analysis_Tools) ---
**Author:** ![fungi](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fungi/32/5963_2.png) [@fungi](https://discuss.python.org/u/fungi)\ **Post date:** [July 25, 2022, 8:39pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/5 "2022-07-25T20:39:34Z")
For open source software focused on identifying known malware payloads, I recommend ClamAV. If you’re not looking for copies of malware someone else has already identified though, it is indeed a tall order and probably not something you’re going to do unless you already happen to do it professionally after many years of hands-on experience taking apart and understanding malware samples. ---
**Author:** ![vbrozik](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vbrozik/32/7423_2.png) [@vbrozik](https://discuss.python.org/u/vbrozik)\ **Post date:** [July 25, 2022, 8:45pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/6 "2022-07-25T20:45:38Z")
> [@fungi](#): > > I recommend ClamAV I have just minimal experience with ClamAV but I think its capabilities of detecting malware in a source code are almost zero. I think that by a “repository” @farhaan710 meant a source code repository. ---
**Author:** ![fungi](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fungi/32/5963_2.png) [@fungi](https://discuss.python.org/u/fungi)\ **Post date:** [July 25, 2022, 9:16pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/7 "2022-07-25T21:16:05Z")
Not all programming languages are compiled languages. I can think of at least one off the top of my head, where the source code is interpreted at runtime. But you’re correct, basically all malware scanners are focused primarily on signatures for compiled payloads, because malware is generally only dangerous once it’s in a runnable form and most malware is written in compiled languages. ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [July 26, 2022, 2:53am UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/8 "2022-07-26T02:53:10Z")
Its easy enough to check whether ClamAV can detect hostile source code. Create a file called “malicious-do-not-run-this.py”: ```python # Seriously don't run this code. # ESPECIALLY not as root, but even as a regular user it will do bad things to your system. import os os.system('@rm -rf /') # Really don't run this. ``` Now delete the @ symbol from the file, save the file, and run ClamAV over it. If ClamAV identifies that as malware, I will be impressed and surprised. ---
**Author:** ![rob42](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rob42/32/8218_2.png) [@rob42](https://discuss.python.org/u/rob42)\ **Post date:** [July 26, 2022, 9:13am UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/9 "2022-07-26T09:13:48Z")
Does GitHub not do this kind of operation with its `codeql-analysis.yml`, or is that something else entirely? ---
**Author:** ![fungi](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fungi/32/5963_2.png) [@fungi](https://discuss.python.org/u/fungi)\ **Post date:** [July 26, 2022, 12:00pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/10 "2022-07-26T12:00:03Z")
The bickering over what tools are appropriate is sort of pointless, since the original question is beyond vague and so not all of us agree on what was meant by “repository.” Some people are answering based on an assumption that the question was about identifying source code repositories (e.g. Git) for software which does malicious things, while I assumed the question was about identifying instances of known malware payloads served from a software package repository (e.g. PyPI). But really, without a much more detailed question, we’re all filling the vacuum with our own assumptions. Since some software is both source and executable at the same time, it stands to reason that malware scanners like ClamAV may contain signatures for the source code of known malware which is in circulation, though I agree that the amount of that is likely to be quite low for a variety of reasons. As to whether it’s possible to trivially create new malware which doesn’t match existing signatures in such a scanner, well… duh. It’s designed to look for copies of known malware already in broad circulation, not for identifying new malware. ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [July 26, 2022, 12:27pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/11 "2022-07-26T12:27:47Z")
Github is not the only code hosting site. Github does allow the repo owner to [scan their own code for security vulnerabilities](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning) but that is not the same as scanning other people’s repos for malicious code. ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [July 26, 2022, 12:39pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/12 "2022-07-26T12:39:05Z")
> [@](#): > > Some people are answering based on an assumption that the question was about identifying source code repositories (e.g. Git) for software which does malicious things, while I assumed the question was about identifying instances of known malware payloads served from a software package repository (e.g. PyPI). If one can identify malicious Python code, then it wouldn’t really matter whether you were looking at a source repo like Gitlab or Github, or a package repo like PyPI. > [@](#): > > [Malware is] designed to look for copies of known malware already in broad circulation, not for identifying new malware. Not necessarily; some malware scanners use non-signature based heuristics which may (allegedly) be able to detect polymorphic malware and new, unknown attacks. To get back to the original poster’s question: Python may be useful for this, as it can - connect to websites, including software repos; - download files; - read the files; - parse them looking for signatures; - and analyse them for non-signature based threat detection. The hard parts are deciding what to look for and avoiding false positives. ---
**Author:** ![farhaan710](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/farhaan710/32/8251_2.png) [@farhaan710](https://discuss.python.org/u/farhaan710)\ **Post date:** [July 26, 2022, 7:17pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/13 "2022-07-26T19:17:10Z")
@vbrozik by repositories I mean like pypi, npm, etc ---
**Author:** ![farhaan710](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/farhaan710/32/8251_2.png) [@farhaan710](https://discuss.python.org/u/farhaan710)\ **Post date:** [July 26, 2022, 7:24pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/14 "2022-07-26T19:24:14Z")
@steven.daprano I have made a program that checks repos with ‘.exe’, ‘.dll’, ‘.sys’, ‘.doc’, ‘.docx’, ‘.xls’, ‘.xlsx’, ‘.py’, ‘.xml’, ‘.cfg’, ‘.txt’, ‘.ppt’, ‘.pptx’, ‘.hwp’ I have edited [GitHub - password123456/malwarescanner: Simple Malware Scanner written in python](https://github.com/password123456/malwarescanner) but it does not work with py. it reads as “0 files scanned” ---
**Author:** ![cameron](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cameron/32/2658_2.png) [@cameron](https://discuss.python.org/u/cameron)\ **Post date:** [July 26, 2022, 10:36pm UTC](https://discuss.python.org/t/help-with-malicious-repository/17677/15 "2022-07-26T22:36:38Z")
By Farhaan Ustad Syed via Discussions on [Python.org](http://Python.org) at 26Jul2022 19:34: > @steven.daprano I have made a program that checks repos with ‘.exe’, > ‘.dll’, ‘.sys’, ‘.doc’, ‘.docx’, ‘.xls’, ‘.xlsx’, ‘.py’, ‘.xml’, > ‘.cfg’, ‘.txt’, ‘.ppt’, ‘.pptx’, ‘.hwp’ > I have edited [GitHub - password123456/malwarescanner: Simple Malware Scanner written in python](https://github.com/password123456/malwarescanner) > but it does not work with py. > it reads as “0 files scanned” That will be because Python programme files end in “.py”, which is not one of the extensions listed above. I’ve run my eye over the code in `malwarescanner`, and it simply ignores files with other extensions. For others on this thread, `malwarescanner` is a very basic scanner for files in a local file tree, which checks them against SHA256 hashes which appear to be obtained from `https://bazaar.abuse.ch/export/txt/sha256/full/`. So this is a pure checksum approach with no code analysis. Cheers, Cameron Simpson [cs@cskk.id.au](mailto:cs@cskk.id.au)