# Making Cinder more broadly available **URL:** \ **Category:** Core Development\ **Created:** [March 4, 2022, 6:34pm UTC](https://discuss.python.org/t/making-cinder-more-broadly-available/14062 "2022-03-04T18:34:42Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Dino](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dino/32/344_2.png) [@Dino](https://discuss.python.org/u/Dino)\ **Post date:** [March 4, 2022, 6:34pm UTC](https://discuss.python.org/t/making-cinder-more-broadly-available/14062/1 "2022-03-04T18:34:42Z")
[re-posting here for visibility, should have done both at the same time!] Hi, [Cinder](https://github.com/facebookincubator/cinder) is Meta’s performance-oriented version of CPython 3.8. It has been in use as the production Python behind Instagram server for years, as well as powering various other Python applications across Meta. We are interested in making the improvements in Cinder more broadly available to the Python community. As a collection of features and components built on top of CPython, we’d like to: 1. Merge the feature into CPython for core features that are highly coupled with CPython internals (after we port it over to the main branch) 2. For features that _can_ work as extension modules (like Cinder JIT) - extract them as a new open source standalone pip-installable extension module. 3. To enable the pip-installable extension module, we will need to add a few extension hooks to CPython, which we intend to carve out and port over to the main branch in order to merge upstream. Please read more about our upstreaming strategy and a few Cinder features in this document: > **[Contributing Cinder](https://docs.google.com/document/d/1l8I-FDE1xrIShm9eSNJqsGmY_VanMDX5-aK_gujhYBI/edit?usp=sharing)** > > Contributing Cinder Cinder is Meta’s internal performance-oriented Python runtime, powering Instagram Server in production, as well as various other internal Python applications across Meta. Cinder is currently based on CPython 3.8.5, and was open... Complete source code of our CPython fork: > **[GitHub - facebookincubator/cinder: Instagram's performance oriented fork of...](https://github.com/facebookincubator/cinder)** > > Instagram's performance oriented fork of CPython. Contribute to facebookincubator/cinder development by creating an account on GitHub. We started filing bpo issues for a few of the pieces we want to merge upstream (links are in the google doc), and are looking forward to feedback from the community on the issues, overall approach, or any feedback on the features themselves. Cheers, Dino ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [March 14, 2022, 2:55pm UTC](https://discuss.python.org/t/making-cinder-more-broadly-available/14062/2 "2022-03-14T14:55:42Z")
Thanks for writing this up! As you note, a lot of this work overlaps with the Faster CPython project, and if you can coordinate with them I don’t see why the parts couldn’t be merged. Pull requests for the bpo issuses are the next step :‍) I wonder how much of the Lazy Imports would be doable using a custom loader with a custom .pyc compilation step, to avoid needing this in core CPython _right_ now. Is that worth considering? (Alas, I haven’t actually looked at the code.) ---
**Author:** ![Dino](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dino/32/344_2.png) [@Dino](https://discuss.python.org/u/Dino)\ **Post date:** [March 14, 2022, 9:25pm UTC](https://discuss.python.org/t/making-cinder-more-broadly-available/14062/3 "2022-03-14T21:25:07Z")
Hi Petr, We’re definitely working with the faster CPython folks and making sure we’re coordinated there and have been opening the bpo’s for the immediate concrete changes! On the lazy imports issue, I think what you’re suggesting is probably possible, but it’s not going to be able to get the abstraction 100% right, and it’s going to take a significant performance hit. The initial implementation of this was actually something which hooked into the various opcodes in the interpreter loop and checked for the deferred objects. And indeed an equivalent implementation could be done at the .pyc level. For example LOAD\_NAME is followed by various opcodes to check if the object is deferred and resolve it rather than embedding that in the interpreter loop. But even with having the implementation in C there were perf regressions using this approach and obviously it’d be much slower in interpreted code. But another problem with it was it was just hard to track down all of the various places where it was possible for the deferred objects to escape. For example it led to having to have logic in LOAD\_FAST to handle deferred objects which is kind of terrible. And fundamentally because the globals are just in a normal un-abstracted dict that is fully exposed once that leaks so do the deferred objects. Both the performance concerns and the escaping concerns led to the maybe dramatic decision to embed the logic in the dictionary object, which actually was performance neutral - and that’s because we could leverage the existing dk\_lookup mechanism to only have any cost for dictionaries which had deferred objects in them. And it solved the problem of deferred objects escaping because it was handled at the source of truth in the dictionary. So the performance concerns aren’t huge if people just wanted to experiment with the feature as an optional add-in, but the leaky abstraction is likely going to lead to a lot more issues with being able to successfully use it in large existing programs. ---
**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\ **Post date:** [March 15, 2022, 9:13am UTC](https://discuss.python.org/t/making-cinder-more-broadly-available/14062/4 "2022-03-15T09:13:12Z")
That makes sense. Thank you for the explanation! ---
**Author:** ![ofek](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ofek/32/1033_2.png) [@ofek](https://discuss.python.org/u/ofek)\ **Post date:** [March 25, 2022, 2:57pm UTC](https://discuss.python.org/t/making-cinder-more-broadly-available/14062/5 "2022-03-25T14:57:41Z")
As the maintainer of several large CLIs, I’m extremely excited for the possibility of lazy imports!