PySRRegressor Reference¶
PySRRegressor
has many options for controlling a symbolic regression search.
Let's look at them below.
PySRRegressor Parameters¶
The Algorithm¶
Creating the Search Space¶

binary_operators
List of strings for binary operators used in the search. See the operators page for more details.
Default:
["+", "", "*", "/"]

unary_operators
Operators which only take a single scalar as input. For example,
"cos"
or"exp"
.Default:
None

maxsize
Max complexity of an equation.
Default:
20

maxdepth
Max depth of an equation. You can use both
maxsize
andmaxdepth
.maxdepth
is by default not used.Default:
None
Setting the Search Size¶

niterations
Number of iterations of the algorithm to run. The best equations are printed and migrate between populations at the end of each iteration.
Default:
40

populations
Number of populations running.
Default:
15

population_size
Number of individuals in each population.
Default:
33

ncycles_per_iteration
Number of total mutations to run, per 10 samples of the population, per iteration.
Default:
550
The Objective¶

elementwise_loss
String of Julia code specifying an elementwise loss function. Can either be a loss from LossFunctions.jl, or your own loss written as a function. Examples of custom written losses include:
myloss(x, y) = abs(xy)
for nonweighted, ormyloss(x, y, w) = w*abs(xy)
for weighted. The included losses include: Regression:LPDistLoss{P}()
,L1DistLoss()
,L2DistLoss()
(mean square),LogitDistLoss()
,HuberLoss(d)
,L1EpsilonInsLoss(ϵ)
,L2EpsilonInsLoss(ϵ)
,PeriodicLoss(c)
,QuantileLoss(τ)
. Classification:ZeroOneLoss()
,PerceptronLoss()
,L1HingeLoss()
,SmoothedL1HingeLoss(γ)
,ModifiedHuberLoss()
,L2MarginLoss()
,ExpLoss()
,SigmoidLoss()
,DWDMarginLoss(q)
.Default:
"L2DistLoss()"

loss_function
Alternatively, you can specify the full objective function as a snippet of Julia code, including any sort of custom evaluation (including symbolic manipulations beforehand), and any sort of loss function or regularizations. The default
loss_function
used in SymbolicRegression.jl is roughly equal to:where the example elementwise loss is meansquared error. You may pass a function with the same arguments as this (note that the name of the function doesn't matter). Here, bothfunction eval_loss(tree, dataset::Dataset{T,L}, options)::L where {T,L} prediction, flag = eval_tree_array(tree, dataset.X, options) if !flag return L(Inf) end return sum((prediction . dataset.y) .^ 2) / dataset.n end
prediction
anddataset.y
are 1D arrays of lengthdataset.n
. If usingbatching
, then you should add anidx
argument to the function, which isnothing
for nonbatched, and a 1D array of indices for batched.Default:
None

model_selection
Model selection criterion when selecting a final expression from the list of best expression at each complexity. Can be
'accuracy'
,'best'
, or'score'
.'accuracy'
selects the candidate model with the lowest loss (highest accuracy).'score'
selects the candidate model with the highest score. Score is defined as the negated derivative of the logloss with respect to complexity  if an expression has a much better loss at a slightly higher complexity, it is preferred.'best'
selects the candidate model with the highest score among expressions with a loss better than at least 1.5x the most accurate model.Default:
'best'

dimensional_constraint_penalty
Additive penalty for if dimensional analysis of an expression fails. By default, this is
1000.0
. 
dimensionless_constants_only
Whether to only search for dimensionless constants, if using units.
Default:
False
Working with Complexities¶

parsimony
Multiplicative factor for how much to punish complexity.
Default:
0.0032

constraints
Dictionary of int (unary) or 2tuples (binary), this enforces maxsize constraints on the individual arguments of operators. E.g.,
'pow': (1, 1)
says that power laws can have any complexity left argument, but only 1 complexity in the right argument. Use this to force more interpretable solutions.Default:
None

nested_constraints
Specifies how many times a combination of operators can be nested. For example,
{"sin": {"cos": 0}}, "cos": {"cos": 2}}
specifies thatcos
may never appear within asin
, butsin
can be nested with itself an unlimited number of times. The second term specifies thatcos
can be nested up to 2 times within acos
, so thatcos(cos(cos(x)))
is allowed (as well as any combination of+
or
within it), butcos(cos(cos(cos(x))))
is not allowed. When an operator is not specified, it is assumed that it can be nested an unlimited number of times. This requires that there is no operator which is used both in the unary operators and the binary operators (e.g.,
could be both subtract, and negation). For binary operators, you only need to provide a single number: both arguments are treated the same way, and the max of each argument is constrained.Default:
None

complexity_of_operators
If you would like to use a complexity other than 1 for an operator, specify the complexity here. For example,
{"sin": 2, "+": 1}
would give a complexity of 2 for each use of thesin
operator, and a complexity of 1 for each use of the+
operator (which is the default). You may specify real numbers for a complexity, and the total complexity of a tree will be rounded to the nearest integer after computing.Default:
None

complexity_of_constants
Complexity of constants.
Default:
1

complexity_of_variables
Global complexity of variables. To set different complexities for different variables, pass a list of complexities to the
fit
method with keywordcomplexity_of_variables
. You cannot use both.Default:
1

warmup_maxsize_by
Whether to slowly increase max size from a small number up to the maxsize (if greater than 0). If greater than 0, says the fraction of training time at which the current maxsize will reach the userpassed maxsize.
Default:
0.0

use_frequency
Whether to measure the frequency of complexities, and use that instead of parsimony to explore equation space. Will naturally find equations of all complexities.
Default:
True

use_frequency_in_tournament
Whether to use the frequency mentioned above in the tournament, rather than just the simulated annealing.
Default:
True

adaptive_parsimony_scaling
If the adaptive parsimony strategy (
use_frequency
anduse_frequency_in_tournament
), this is how much to (exponentially) weight the contribution. If you find that the search is only optimizing the most complex expressions while the simpler expressions remain stagnant, you should increase this value.Default:
20.0

should_simplify
Whether to use algebraic simplification in the search. Note that only a few simple rules are implemented.
Default:
True
Mutations¶

weight_add_node
Relative likelihood for mutation to add a node.
Default:
0.79

weight_insert_node
Relative likelihood for mutation to insert a node.
Default:
5.1

weight_delete_node
Relative likelihood for mutation to delete a node.
Default:
1.7

weight_do_nothing
Relative likelihood for mutation to leave the individual.
Default:
0.21

weight_mutate_constant
Relative likelihood for mutation to change the constant slightly in a random direction.
Default:
0.048

weight_mutate_operator
Relative likelihood for mutation to swap an operator.
Default:
0.47

weight_swap_operands
Relative likehood for swapping operands in binary operators.
Default:
0.1

weight_randomize
Relative likelihood for mutation to completely delete and then randomly generate the equation
Default:
0.00023

weight_simplify
Relative likelihood for mutation to simplify constant parts by evaluation
Default:
0.0020

weight_optimize
Constant optimization can also be performed as a mutation, in addition to the normal strategy controlled by
optimize_probability
which happens every iteration. Using it as a mutation is useful if you want to use a largencycles_periteration
, and may not optimize very often.Default:
0.0

crossover_probability
Absolute probability of crossovertype genetic operation, instead of a mutation.
Default:
0.066

annealing
Whether to use annealing.
Default:
False

alpha
Initial temperature for simulated annealing (requires
annealing
to beTrue
).Default:
0.1

perturbation_factor
Constants are perturbed by a max factor of (perturbation_factor*T + 1). Either multiplied by this or divided by this.
Default:
0.076

skip_mutation_failures
Whether to skip mutation and crossover failures, rather than simply resampling the current member.
Default:
True
Tournament Selection¶

tournament_selection_n
Number of expressions to consider in each tournament.
Default:
10

tournament_selection_p
Probability of selecting the best expression in each tournament. The probability will decay as p*(1p)^n for other expressions, sorted by loss.
Default:
0.86
Constant Optimization¶

optimizer_algorithm
Optimization scheme to use for optimizing constants. Can currently be
NelderMead
orBFGS
.Default:
"BFGS"

optimizer_nrestarts
Number of time to restart the constants optimization process with different initial conditions.
Default:
2

optimize_probability
Probability of optimizing the constants during a single iteration of the evolutionary algorithm.
Default:
0.14

optimizer_iterations
Number of iterations that the constants optimizer can take.
Default:
8

should_optimize_constants
Whether to numerically optimize constants (NelderMead/Newton) at the end of each iteration.
Default:
True
Migration between Populations¶

fraction_replaced
How much of population to replace with migrating equations from other populations.
Default:
0.000364

fraction_replaced_hof
How much of population to replace with migrating equations from hall of fame.
Default:
0.035

migration
Whether to migrate.
Default:
True

hof_migration
Whether to have the hall of fame migrate.
Default:
True

topn
How many top individuals migrate from each population.
Default:
12
Data Preprocessing¶

denoise
Whether to use a Gaussian Process to denoise the data before inputting to PySR. Can help PySR fit noisy data.
Default:
False

select_k_features
Whether to run feature selection in Python using random forests, before passing to the symbolic regression code. None means no feature selection; an int means select that many features.
Default:
None
Stopping Criteria¶

max_evals
Limits the total number of evaluations of expressions to this number.
Default:
None

timeout_in_seconds
Make the search return early once this many seconds have passed.
Default:
None

early_stop_condition
Stop the search early if this loss is reached. You may also pass a string containing a Julia function which takes a loss and complexity as input, for example:
"f(loss, complexity) = (loss < 0.1) && (complexity < 10)"
.Default:
None
Performance and Parallelization¶

procs
Number of processes (=number of populations running).
Default:
cpu_count()

multithreading
Use multithreading instead of distributed backend. Using procs=0 will turn off both.
Default:
True

cluster_manager
For distributed computing, this sets the job queue system. Set to one of "slurm", "pbs", "lsf", "sge", "qrsh", "scyld", or "htc". If set to one of these, PySR will run in distributed mode, and use
procs
to figure out how many processes to launch.Default:
None

heap_size_hint_in_bytes
For multiprocessing, this sets the
heapsizehint
parameter for new Julia processes. This can be configured when using multinode distributed compute, to give a hint to each process about how much memory they can use before aggressive garbage collection. 
batching
Whether to compare population members on small batches during evolution. Still uses full dataset for comparing against hall of fame.
Default:
False

batch_size
The amount of data to use if doing batching.
Default:
50

precision
What precision to use for the data. By default this is
32
(float32), but you can select64
or16
as well, giving you 64 or 16 bits of floating point precision, respectively. If you pass complex data, the corresponding complex precision will be used (i.e.,64
for complex128,32
for complex64).Default:
32

fast_cycle
Batch over population subsamples. This is a slightly different algorithm than regularized evolution, but does cycles 15% faster. May be algorithmically less efficient.
Default:
False

turbo
(Experimental) Whether to use LoopVectorization.jl to speed up the search evaluation. Certain operators may not be supported. Does not support 16bit precision floats.
Default:
False

bumper
(Experimental) Whether to use Bumper.jl to speed up the search evaluation. Does not support 16bit precision floats.
Default:
False

enable_autodiff
Whether to create derivative versions of operators for automatic differentiation. This is only necessary if you wish to compute the gradients of an expression within a custom loss function.
Default:
False
Determinism¶

random_state
Pass an int for reproducible results across multiple function calls. See :term:
Glossary <random_state>
.Default:
None

deterministic
Make a PySR search give the same result every run. To use this, you must turn off parallelism (with
procs
=0,multithreading
=False), and setrandom_state
to a fixed seed.Default:
False

warm_start
Tells fit to continue from where the last call to fit finished. If false, each call to fit will be fresh, overwriting previous results.
Default:
False
Monitoring¶

verbosity
What verbosity level to use. 0 means minimal print statements.
Default:
1

update_verbosity
What verbosity level to use for package updates. Will take value of
verbosity
if not given.Default:
None

print_precision
How many significant digits to print for floats.
Default:
5

progress
Whether to use a progress bar instead of printing to stdout.
Default:
True
Environment¶

temp_equation_file
Whether to put the hall of fame file in the temp directory. Deletion is then controlled with the
delete_tempfiles
parameter.Default:
False

tempdir
directory for the temporary files.
Default:
None

delete_tempfiles
Whether to delete the temporary files after finishing.
Default:
True

update
Whether to automatically update Julia packages when
fit
is called. You should make sure that PySR is uptodate itself first, as the packaged Julia packages may not necessarily include all updated dependencies.Default:
False
Exporting the Results¶

equation_file
Where to save the files (.csv extension).
Default:
None

output_jax_format
Whether to create a 'jax_format' column in the output, containing jaxcallable functions and the default parameters in a jax array.
Default:
False

output_torch_format
Whether to create a 'torch_format' column in the output, containing a torch module with trainable parameters.
Default:
False

extra_sympy_mappings
Provides mappings between custom
binary_operators
orunary_operators
defined in julia strings, to those same operators defined in sympy. E.G ifunary_operators=["inv(x)=1/x"]
, then for the fitted model to be export to sympy,extra_sympy_mappings
would be{"inv": lambda x: 1/x}
.Default:
None

extra_torch_mappings
The same as
extra_jax_mappings
but for model export to pytorch. Note that the dictionary keys should be callable pytorch expressions. For example:extra_torch_mappings={sympy.sin: torch.sin}
.Default:
None

extra_jax_mappings
Similar to
extra_sympy_mappings
but for model export to jax. The dictionary maps sympy functions to jax functions. For example:extra_jax_mappings={sympy.sin: "jnp.sin"}
maps thesympy.sin
function to the equivalent jax expressionjnp.sin
.Default:
None
PySRRegressor Functions¶
fit(X, y, Xresampled=None, weights=None, variable_names=None, complexity_of_variables=None, X_units=None, y_units=None)
¶
Search for equations to fit the dataset and store them in self.equations_
.
Parameters:
Name  Type  Description  Default 

X 
ndarray  DataFrame

Training data of shape (n_samples, n_features). 
required 
y 
ndarray  DataFrame

Target values of shape (n_samples,) or (n_samples, n_targets). Will be cast to X's dtype if necessary. 
required 
Xresampled 
ndarray  DataFrame

Resampled training data, of shape (n_resampled, n_features),
to generate a denoised data on. This
will be used as the training data, rather than 
None

weights 
ndarray  DataFrame

Weight array of the same shape as 
None

variable_names 
list[str]

A list of names for the variables, rather than "x0", "x1", etc.
If 
None

X_units 
list[str]

A list of units for each variable in 
None

y_units 
str  list[str]

Similar to 
None

Returns:
Name  Type  Description 

self 
object

Fitted estimator. 
Source code in pysr/sr.py
1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 

predict(X, index=None)
¶
Predict y from input X using the equation chosen by model_selection
.
You may see what equation is used by printing this object. X should have the same columns as the training data.
Parameters:
Name  Type  Description  Default 

X 
ndarray  DataFrame

Training data of shape 
required 
index 
int  list[int]

If you want to compute the output of an expression using a
particular row of 
None

Returns:
Name  Type  Description 

y_predicted 
ndarray of shape (n_samples, nout_)

Values predicted by substituting 
Raises:
Type  Description 

ValueError

Raises if the 
Source code in pysr/sr.py
2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 

from_file(equation_file, *, binary_operators=None, unary_operators=None, n_features_in=None, feature_names_in=None, selection_mask=None, nout=1, **pysr_kwargs)
classmethod
¶
Create a model from a saved model checkpoint or equation file.
Parameters:
Name  Type  Description  Default 

equation_file 
str or Path

Path to a pickle file containing a saved model, or a csv file containing equations. 
required 
binary_operators 
list[str]

The same binary operators used when creating the model. Not needed if loading from a pickle file. 
None

unary_operators 
list[str]

The same unary operators used when creating the model. Not needed if loading from a pickle file. 
None

n_features_in 
int

Number of features passed to the model. Not needed if loading from a pickle file. 
None

feature_names_in 
list[str]

Names of the features passed to the model. Not needed if loading from a pickle file. 
None

selection_mask 
NDArray[bool_]

If using 
None

nout 
int

Number of outputs of the model.
Not needed if loading from a pickle file.
Default is 
1

**pysr_kwargs 
dict

Any other keyword arguments to initialize the PySRRegressor object. These will overwrite those stored in the pickle file. Not needed if loading from a pickle file. 
{}

Returns:
Name  Type  Description 

model 
PySRRegressor

The model with fitted equations. 
Source code in pysr/sr.py
949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 

sympy(index=None)
¶
Return sympy representation of the equation(s) chosen by model_selection
.
Parameters:
Name  Type  Description  Default 

index 
int  list[int]

If you wish to select a particular equation from

None

Returns:
Name  Type  Description 

best_equation 
str, list[str] of length nout_

SymPy representation of the best equation. 
Source code in pysr/sr.py
latex(index=None, precision=3)
¶
Return latex representation of the equation(s) chosen by model_selection
.
Parameters:
Name  Type  Description  Default 

index 
int  list[int]

If you wish to select a particular equation from

None

precision 
int

The number of significant figures shown in the LaTeX
representation.
Default is 
3

Returns:
Name  Type  Description 

best_equation 
str or list[str] of length nout_

LaTeX expression of the best equation. 
Source code in pysr/sr.py
pytorch(index=None)
¶
Return pytorch representation of the equation(s) chosen by model_selection
.
Each equation (multiple given if there are multiple outputs) is a PyTorch module
containing the parameters as trainable attributes. You can use the module like
any other PyTorch module: module(X)
, where X
is a tensor with the same
column ordering as trained with.
Parameters:
Name  Type  Description  Default 

index 
int  list[int]

If you wish to select a particular equation from

None

Returns:
Name  Type  Description 

best_equation 
Module

PyTorch module representing the expression. 
Source code in pysr/sr.py
jax(index=None)
¶
Return jax representation of the equation(s) chosen by model_selection
.
Each equation (multiple given if there are multiple outputs) is a dictionary
containing {"callable": func, "parameters": params}. To call func
, pass
func(X, params). This function is differentiable using jax.grad
.
Parameters:
Name  Type  Description  Default 

index 
int  list[int]

If you wish to select a particular equation from

None

Returns:
Name  Type  Description 

best_equation 
dict[str, Any]

Dictionary of callable jax function in "callable" key, and jax array of parameters as "parameters" key. 
Source code in pysr/sr.py
latex_table(indices=None, precision=3, columns=['equation', 'complexity', 'loss', 'score'])
¶
Create a LaTeX/booktabs table for all, or some, of the equations.
Parameters:
Name  Type  Description  Default 

indices 
list[int]  list[list[int]]

If you wish to select a particular subset of equations from

None

precision 
int

The number of significant figures shown in the LaTeX
representations.
Default is 
3

columns 
list[str]

Which columns to include in the table.
Default is 
['equation', 'complexity', 'loss', 'score']

Returns:
Name  Type  Description 

latex_table_str 
str

A string that will render a table in LaTeX of the equations. 
Source code in pysr/sr.py
refresh(checkpoint_file=None)
¶
Update self.equations_ with any new options passed.
For example, updating extra_sympy_mappings
will require a .refresh()
to update the equations.
Parameters:
Name  Type  Description  Default 

checkpoint_file 
str or Path

Path to checkpoint hall of fame file to be loaded.
The default will use the set 
None
