How do I make a version 18 magic.mgc file?

I’ve recently taken over the Django backend and it seems some things are out of date. In particular, I can’t get past several commands because my magic.mgc file is out of date. Currently I’m trying to run makemigrations, and I’m getting this error:

a couple hundred lines simiar to:

app_common/misc/magic.mgc, 2617: Warning: offset `.' invalid

and then the stack trace ending in the following error message:

magic.MagicException: b"File 5.45 supports only version 18 magic files. `app_common/misc/magic.mgc' is version 14"

I could reproduce the stack trace, but it basically boils down to having an out of date magic file in the project. I have tried creating a new one, but the highest level I can get File to create is version 16. I’ve noticed that file -v returns version 5.41 instead of 5.45 which likely is why file is creating the wrong file. But this seems to be a mac system file and I don’t know how to use the updated file 5.45 to create the magic file. I also have tried installing older versions of python-magic, but have not been able to find a previous version of libmagic to go back to that would support 14 or even 16 magic.mgc files.

Does anyone know how I could fix this?

Is there any way to either

My guess is that you are not using versions of file and libmagic that are compatible.
Which OS are you doing this on? I think you hint its macOS.
Are you using a mix of homebrew and macOS files?

Maybe. I’m not doing that on purpose, but I’m suspecting thats whats happening. I suspect that becasue it seems that file 5.41 supports magic files of version 16, which is what I create when I run file -C -m magic.mgc app_common/misc/ even though I have downloaded file 5.45. Honestly, help in how to not be using a mix would be great :confused:

If you are downloading libmagic to get a newer version then also download and build the file command that matches.

Or you could install homebrew and use file and libmagic from them.

I do have homebrew and that was where I installed file and libmagic from. I think where the mixing is coming in is that the system version of file is staying at 5.41 even though I’ve installed 5.45 through homebrew. I could be wrong though and maybe its something else I’m doing. Every time I’ve tried building the file again, I always only get a version 16 magic file, which I think maybe is coming from the system version, not the one I’ve installed through homebrew. Is it possible to specify that I want the one I installed to build the file instead of the macOS version?

I’m sorry If I’m not using the right terms or descriptions, by the way. I am somewhat new to Django and backend dev and Python too :confused:

One way is to use the full path to the file command.
Another is to set the PATH so that the homebrew bin folder is at the front of the PATH.
Also you can check where commands comes from using the which command.

Ok, I like that first idea. How would I use the full path to the file command from Homebrew? You saying that and suspecting that the conflict is a MacOS version of file utility makes me think maybe I could rename the homebrew command to file2 so that the macOS can still use it’s file command using the version it includes and I could use the homebrew version as file2. Do you think this would be a good solution?

What I did to find the details for you. I guessed that I should be able to install “file”.

brew install file

This does install libmagic and the file command.
But its not in the usual place! (No idea why not)

Its in $HOME/homebrew/opt/file-formula/bin/file.
If you use that path to compile your magic.mgc it will use the libmagic installed by homebrew.

FYI: On macOS you can use otool -L to see which share libraries a command or library uses (This is like the linux ldd command).

% otool -L /Users/barry/homebrew/opt/file-formula/bin/file
/Users/barry/homebrew/opt/file-formula/bin/file:
	@loader_path/../../../../opt/libmagic/lib/libmagic.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
	/usr/lib/liblzma.5.dylib (compatibility version 6.0.0, current version 6.3.0)
	/usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.8)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)

There is no need to rename commands. Just use the full path.
Also you will likely mess things up for updates if you rename files.

I think I’m doing something wrong with the command. I brew uninstalled and reinstalled file just to make sure I had a clean start (I had thought to try aliases, so I also removed the alias). And the path homebrew gives me is

/opt/homebrew/Cellar/file-formula/5.45/bin

Doesn’t look quite the same as yours, but when I CD to that location I do see file in there, so I think it’s ok. The command I’m using to create the magic file is:

/opt/homebrew/Cellar/file-formula/5.45/bin/file -C -m magic.mgc myapp/misc/

The response I’m getting is:

file: could not find any valid magic files!

I’m thinking I might not have it quite right. Though, I think this was the command I had used before that was creating the version 16 with the system file utility.

I really appreciate you helping by the way! I’d been getting frustrated, but I feel like this is finally on the right track! :slight_smile:

So, I think I figure out what I was doing wrong this time. I didn’t have a name for the magic file and I did just see that there was a “.mgc” file. I renamed that to magic.mgc and now I’m getting different errors when I run makemigrations. So I think that might have fixed it!!

You’re a genius :slight_smile:

Now I need to fix my other issues and maybe I can actually get this thing working :slight_smile:

Just so you know, that did work. The errors had to do with the fact that I’m using Python 3.11 and the previous dev was using 3.6 and several things are in Collections.abc rather than Collections now. But I got past all that and now it ran makemigrations w/o errors :slight_smile: Just wanted to let you know it was for sure successful :slight_smile:

2 Likes

I recall that this is a macOS intel vs. macOS apple silicon difference.