What exactly is the resolution for the probability distributions? Is it an approach that also works in other situations…
We just added scipy.stats.exp and scipy.stats.log for this release. If some sort of standard comes into being, we could support it.
Regarding probability distributions (the topic of this thread), they are most likely themselves going to be arrays (in the sense that they have a shape, support indexing, etc.).
The random variables accept array shape parameters, so internally they have a _shape (which I’ve considered making public), and I’ve suggested that they could support indexing to isolate elements/slices. Currently, the parameters must by NumPy arrays, but all the underlying routines for quadrature, root finding, minimization, etc. have been translated to the Array API, so I hope to add full Array API support in the next release.
But they will only have a small subset of the required attributes/methods of the Array API, and there will not be a corresponding Array API namespace for them.
To be clear for anyone else reading this these are methods like p.exp() and p.log(). That seems like a reasonable approach as a stop-gap right now.
A more general solution is needed for these situations though that is not just adding ad-hoc methods like this because these methods are not present on e.g. int and float. The problem remains that there should be a way to compute exp(x) for an arbitrary object x. If such a method existed then it would be the obvious answer to the question here.
To be clear for anyone else reading this these are methods like p.exp() and p.log()
W.R.T. SciPy random variables, I meant that we added functions to the scipy.stats namespace that accept a random variable X, like scipy.stats.exp(X); they are not methods called like X.exp(). See examples at the links.
(I agree that it would be better to have a more general solution, though. Regardless of where it is - pretty clear that it won’t be in standard library math - it would be nice for there to be an exp function that works on any object that fulfills some requriement, such as implementing an __exp__ method.)
Disclaimer: I likely don’t understand the array API well enough to contribute here but… oh well!
Your comments about putting scalars in arrays to apply math functions are concerning. There are many cases where I have a single scalar x (of some 3rd party type) and I want to calculate the sin of that scalar. Are you suggesting that every time I do this I should first do
import numpy as np
from third_party_scalers import Scalar
x = Scalar(...)
x = np.array(x)
sinx = np.sin(x)
or similar? The point is that you might want to do math operations on Scalar, but you DON’T want to import numpy.
I guess my goal is to be able to do something like
from third_party_scalars import Scalar
from some_math_package import sin
x = Scalar(...)
sinx = sin(x)
and get sensible results regardless of which third_party_scalars package or Scalar object I am using. some_math_package might be the built-in math package… but maybe not since that package is apparently dedicated to math for float scalars.
Have I correctly captured the goal with my code snippet? Or am I misunderstanding something?
I think what Neil is suggesting is that you should subclass scalar like:
class NdimScalar(Scalar):
...
Then you should implement N-dimensional arrays and all the hundreds of methods and functions that are described in the array API. Then when you’re done doing that you can compute the sin of your scalar.
Honestly, if you want a general set of functions, I think the nicest solution would be doing some kind of dispatch like Oscar is suggesting.
However, I hope that if we go that route, we’ll follow the Array API’s interface closely in terms of:
invariants,
parameter names,
function organization,
type promotion rules,
function-vs-method decisions,
the way context is rolled into the arrays themselves,
etc.
I think the Array API has a well-polished and well-thought-out interface. NumPy, by comparison, has way too many methods, parameters, multiple ways of doing the same thing. I think this is where most of my apprehensiveness comes from. Also, I hate the idea of having to learn multiple standards just because two groups of implementers happen to have different tastes.
Ideally, such a generic math library would not be in the standard library so that it can adapt quickly, and so that mistakes can be fixed without forcing the world to adapt to them.
The time cost of implementing such a dispatch library would have to be worth it for the benefit of being able to mix arrays, distributions, and scalars. It’s probably a good long-term thing, but are we sure people really need it? I think we may as well wait until there’s at least three adopters (scalars, Array API adopters, and SciPy distributions). Scalars are already easy enough to use with the Array API. And SciPy distributions only have two functions for now.
More seriously what is your scalar? Is it something that converts to a float or an int? How are you computing the sin, with math.sin or numpy.sin or something else?
I suspect that you don’t need to worry about any of the things covered in this discussion. If what you have works then don’t worry. This discussion is about things that don’t currently work.
I agree that it should not be in the stdlib. Eventually though I would like it to land there.
As for do people need it I think that many people don’t need it directly. The real benefit of these kinds of standardisation efforts often manifests behind the scenes without people realising how fundamental it was that it suddenly became possible to compute the exp of any object which then made it possible for their generic code to be evaluated, compiled in any way etc.
My scalar is UFloat from uncertainties. Right now I would do
from uncertainties import ufloat
from uncertainties.umath import sin
x = ufloat(1, 0.1)
sinx = sin(x)
(Note for anyone trying to read or run these code snippets that. for historical reasons,ufloat is a simple function that constructs UFloat)
The annoyance is having to import from, yet another, math function package. I’ve figured out that if I do something like
class UFloat:
...
def sin(self):
...
Then I can successfully do
from uncertainties import ufloat
import numpy as np
x = ufloat(1, 0.1)
sinx = np.sin(x)
and the results I want but e.g.
from uncertainties import ufloat
from math import sin
x = ufloat(1, 0.1)
sinx = sin(x)
does not work.
What I desire (basically repeat of what I said above is)
from uncertainties import ufloat
from lightweight_math_functions import sin
x = ufloat(1, 0.1)
sinx = sin(x)
The built-in math module would be great (from my perspective) for lightweight_math_functions but I’m not sure if that is possible. numpy is ok, but it’s too heavy-weight. There’s a chance users might want to use uncertainties without numpy. But for pragmatism sake I’ll point out that it possibly but unlikely that users will use uncertainties but not also have numpy installed.
Ok, that tracks with what I’ve been following from the discussion. Thank you. So I guess I’m +1 on the idea of a dispatch library with some standardized API. That would be nice. I guess the idea is library authors would register their new types with the dispatch library and then the dispatch library would know where to send calls?
I guess it was mentioned above that there would be some overhead associated with the dispatching. So perhaps for, e.g. uncertainties, we could advise users that they can use this dispatch library for convenience if they like, but, we could continue to expose umath so that they can get access to the functions directly to circumvent the dispatching overhead from math_dispatch_package? In the specific case of uncertainties there is a quite a lot of “python work” going on for each umath function call. At least a lot more than type check + dict lookup or something. so I highly doubt the dispatch overhead will impact performance materially, but I’m not sure.
Exactly how this should work is an important unresolved question Registering types would not be my suggested solution but I think that @mikeshardmind suggested it above. Honestly I didn’t understand Mike’s suggestion so I can’t really critique it.
I spelled out my suggestion with explicit contexts above. That might not be palatable but if disregarding convenience then I think it is the proper approach. Disregarding convenience is of course questionable though.
I read this post but don’t fully understand it. It does sound like under your “explicit contexts” suggestion it would be possible to do
from uncertainties import ufloat
from generalmath import sin
x = ufloat(1.0, 0.1)
sinx = sin(x)
Is that correct?
If so, then is it correct that your “explicit contexts” proposal is a strategy to make the above snippet possible that is an alternative to the “dispatch” proposal? If so then my issue that I just don’t have enough technical understand to know what exactly is meant by the technical terms “dispatch” and “explicit contexts” (I have some ideas about it, but not enough for me to fully understand your post)
My apologies for dragging folks through me trying to learn something. I do have some hopes though that my non-expert series of questions/learnings will be helpful to other readers of this thread. Happy to stop my questioning here and just do more research on my own if that would be more appropriate.
I think that it should be possible to make this work. I think it requires in some way that ufloat is going to provide a context object via some special method analogous to the way that the array API retrieves a namespace from an array.
An important question though is this: does ufloat support all of the mathematical functions that could be used if x was an int or a float or an ndarray of floats and if we were using any of the functions from either the math or cmath modules or from numpy or scipy.special? How about all of the functions that are defined in mpmath or arb or sympy?
What if instead of sin I were to use this function:
from genericmath import hyp2f1
x = ufloat(1.0, 0.1)
y = hyp2f1(x, x, x, x)
I’m assuming that the answer is that the uncertainties package and ufloat cannot handle the hyp2f1 function which is fine because it is a less commonly used function.
The question though is how in a typing sense can we communicate what operations ufloat does support and how can generic code be typed so that a type checker can understand that the code is or is not valid for ufloat?
I ask this question from a typing perspective but really this is not only about typing but more generally about how generic code like this can be well defined. If it is well defined then we should be able to express that with types somehow.
I think that the best way for well defined generic code is that there is a context object but there is also a need to be able to call sin(x) without a context object. Typically well typed generic library code should use the context object explicitly like:
The TransContext type that I refer to here is an abstract type that would be defined somewhere (in the genericmath library?) like:
from typing import Protocol
class Context[T](Protocol):
...
class TransContext[T](Context[T], Protocol):
"""Common transcendental functions"""
def sin(x: T) -> T: ...
def cos(x: T) -> T: ...
...
This makes it possible to use the context object explicitly:
def some_func[T: Real](x: T, ctx: TransContext[T]) -> T:
return ctx.sin(x)
import math
from uncertainties import unumpy
some_func(1, math)
some_func(ufloat(1,2), unumpy)
Of course we would also like to have a generic sin function that can be used for any object without a context but there is still a need for the context object to be in the picture somewhere because there needs to be a way for us to express which functions (sin, cos, etc) are supported for which types (ufloat, float etc) and obviously we need to be able to get the actual functions at runtime.
Your ufloat type needs to provide the context:
from uncertainties import unumpy
class ufloat:
def __get_context__(self) -> TransContext[ufloat]:
return unumpy
Then the generic math library can have a sin function and it looks like:
from genericmath import sin
from uncertainties import ufloat
y = sin(ufloat(1,2))
We can be much more specific than TransContext with things like HasSinFunction, HasCosFunction etc. Mostly though it will be more useful to be granular about this and to define commonly useful sets of functions as context subtypes. Those can give a clear target for what full sets of functions it is useful that a library like uncertainties should implement in order to facilitate downstream generic code across multiple libraries/types.
One thing to note is that sin(1) needs some special handling. The int type does not provide the __get_context__() method. I think genericmath should provide special case context objects for int and float using the math module giving it a well typed generic interface that can be interchangeable with other math modules/contexts. The question though is does genericmath.sin emulate __get_context__ for int and float by using those?
An awkwardness also arises when there are multiple arguments. Which of these atan2 functions should be used?
Ok, thank you so much for that thorough explanation building on my code snippet/example. I understand it much better now.
It sounds like what you are calling a “context” could also be called a “namespace”. Basically if a cooperating scalar of type T needs to provide a namespace (via __get_context__) that holds cooperating math functions which are compatible with that type. Then the generic library can hook into this namespace so that the generic math library math functions can work on this type of scalar.
By contrast, a dispatch approach would look like: The generic math library has either a lookup table that maps types to namespaces that include the relevant math functions on that type or for each math function it has a lookup table that maps types to the corresponding math function for that type. The library registers the type in these lookup tables so that at runtime the type can be inspected and the right function can be dispatched.
A third way that has been brought up in this thread would be to define methods on the scalar class corresponding to the math functions. e.g. ufloat would define ufloat.sin. This is similar to the context approach because that generic math library can look for the relevant function on whatever object it has. But yes, I can see that is not as nicely extensible.
I’ll need to read through the posts above again to understand the pros/cons for the context approach vs. the dispatch approach.
I follow also your thoughts questions and discussions further down in the post.
In the array API it is called a namespace. I am making the distinction because in general for something like the decimal module or mpmath etc it is not just a namespace but rather a object that holds some data/state. It has mutable attributes like precision, rounding mode etc. Once you have a context object it takes the role of the namespace because you want to access functions from the context object. It would make sense to have contexts for these things in e.g. numpy as well but instead numpy uses global options (np.seterr).
Complications are needed both in types and runtime to make it so that the generic call sin(1) does the right thing. Probably genericmath needs to provide a math_context that provides the math-module functions. Then also in each function there needs to be a decision about how exactly to handle types like int and float that don’t have the __get_context__ method. If explicit contexts are passed like some_func(1, math) then there is no problem.