PySRRegressor Reference¶
PySRRegressor
has many options for controlling a symbolic regression search.
Let's look at them below.
PySRRegressor Parameters¶
The Algorithm¶
Creating the Search Space¶
-
binary_operators
List of strings for binary operators used in the search. See the operators page for more details.
Default:
["+", "-", "*", "/"]
-
unary_operators
Operators which only take a single scalar as input. For example,
"cos"
or"exp"
.Default:
None
-
maxsize
Max complexity of an equation.
Default:
20
-
maxdepth
Max depth of an equation. You can use both
maxsize
andmaxdepth
.maxdepth
is by default not used.Default:
None
Setting the Search Size¶
-
niterations
Number of iterations of the algorithm to run. The best equations are printed and migrate between populations at the end of each iteration.
Default:
40
-
populations
Number of populations running.
Default:
15
-
population_size
Number of individuals in each population.
Default:
33
-
ncyclesperiteration
Number of total mutations to run, per 10 samples of the population, per iteration.
Default:
550
The Objective¶
-
loss
String of Julia code specifying an elementwise loss function. Can either be a loss from LossFunctions.jl, or your own loss written as a function. Examples of custom written losses include:
myloss(x, y) = abs(x-y)
for non-weighted, ormyloss(x, y, w) = w*abs(x-y)
for weighted. The included losses include: Regression:LPDistLoss{P}()
,L1DistLoss()
,L2DistLoss()
(mean square),LogitDistLoss()
,HuberLoss(d)
,L1EpsilonInsLoss(ϵ)
,L2EpsilonInsLoss(ϵ)
,PeriodicLoss(c)
,QuantileLoss(τ)
. Classification:ZeroOneLoss()
,PerceptronLoss()
,L1HingeLoss()
,SmoothedL1HingeLoss(γ)
,ModifiedHuberLoss()
,L2MarginLoss()
,ExpLoss()
,SigmoidLoss()
,DWDMarginLoss(q)
.Default:
"L2DistLoss()"
-
full_objective
Alternatively, you can specify the full objective function as a snippet of Julia code, including any sort of custom evaluation (including symbolic manipulations beforehand), and any sort of loss function or regularizations. The default
full_objective
used in SymbolicRegression.jl is roughly equal to:where the example elementwise loss is mean-squared error. You may pass a function with the same arguments as this (note that the name of the function doesn't matter). Here, bothfunction eval_loss(tree, dataset::Dataset{T,L}, options)::L where {T,L} prediction, flag = eval_tree_array(tree, dataset.X, options) if !flag return L(Inf) end return sum((prediction .- dataset.y) .^ 2) / dataset.n end
prediction
anddataset.y
are 1D arrays of lengthdataset.n
.Default:
None
-
model_selection
Model selection criterion when selecting a final expression from the list of best expression at each complexity. Can be
'accuracy'
,'best'
, or'score'
.'accuracy'
selects the candidate model with the lowest loss (highest accuracy).'score'
selects the candidate model with the highest score. Score is defined as the negated derivative of the log-loss with respect to complexity - if an expression has a much better loss at a slightly higher complexity, it is preferred.'best'
selects the candidate model with the highest score among expressions with a loss better than at least 1.5x the most accurate model.Default:
'best'
Working with Complexities¶
-
parsimony
Multiplicative factor for how much to punish complexity.
Default:
0.0032
-
constraints
Dictionary of int (unary) or 2-tuples (binary), this enforces maxsize constraints on the individual arguments of operators. E.g.,
'pow': (-1, 1)
says that power laws can have any complexity left argument, but only 1 complexity in the right argument. Use this to force more interpretable solutions.Default:
None
-
nested_constraints
Specifies how many times a combination of operators can be nested. For example,
{"sin": {"cos": 0}}, "cos": {"cos": 2}}
specifies thatcos
may never appear within asin
, butsin
can be nested with itself an unlimited number of times. The second term specifies thatcos
can be nested up to 2 times within acos
, so thatcos(cos(cos(x)))
is allowed (as well as any combination of+
or-
within it), butcos(cos(cos(cos(x))))
is not allowed. When an operator is not specified, it is assumed that it can be nested an unlimited number of times. This requires that there is no operator which is used both in the unary operators and the binary operators (e.g.,-
could be both subtract, and negation). For binary operators, you only need to provide a single number: both arguments are treated the same way, and the max of each argument is constrained.Default:
None
-
complexity_of_operators
If you would like to use a complexity other than 1 for an operator, specify the complexity here. For example,
{"sin": 2, "+": 1}
would give a complexity of 2 for each use of thesin
operator, and a complexity of 1 for each use of the+
operator (which is the default). You may specify real numbers for a complexity, and the total complexity of a tree will be rounded to the nearest integer after computing.Default:
None
-
complexity_of_constants
Complexity of constants.
Default:
1
-
complexity_of_variables
Complexity of variables.
Default:
1
-
warmup_maxsize_by
Whether to slowly increase max size from a small number up to the maxsize (if greater than 0). If greater than 0, says the fraction of training time at which the current maxsize will reach the user-passed maxsize.
Default:
0.0
-
use_frequency
Whether to measure the frequency of complexities, and use that instead of parsimony to explore equation space. Will naturally find equations of all complexities.
Default:
True
-
use_frequency_in_tournament
Whether to use the frequency mentioned above in the tournament, rather than just the simulated annealing.
Default:
True
-
adaptive_parsimony_scaling
If the adaptive parsimony strategy (
use_frequency
anduse_frequency_in_tournament
), this is how much to (exponentially) weight the contribution. If you find that the search is only optimizing the most complex expressions while the simpler expressions remain stagnant, you should increase this value.Default:
20.0
-
should_simplify
Whether to use algebraic simplification in the search. Note that only a few simple rules are implemented.
Default:
True
Mutations¶
-
weight_add_node
Relative likelihood for mutation to add a node.
Default:
0.79
-
weight_insert_node
Relative likelihood for mutation to insert a node.
Default:
5.1
-
weight_delete_node
Relative likelihood for mutation to delete a node.
Default:
1.7
-
weight_do_nothing
Relative likelihood for mutation to leave the individual.
Default:
0.21
-
weight_mutate_constant
Relative likelihood for mutation to change the constant slightly in a random direction.
Default:
0.048
-
weight_mutate_operator
Relative likelihood for mutation to swap an operator.
Default:
0.47
-
weight_randomize
Relative likelihood for mutation to completely delete and then randomly generate the equation
Default:
0.00023
-
weight_simplify
Relative likelihood for mutation to simplify constant parts by evaluation
Default:
0.0020
-
weight_optimize
Constant optimization can also be performed as a mutation, in addition to the normal strategy controlled by
optimize_probability
which happens every iteration. Using it as a mutation is useful if you want to use a largencyclesperiteration
, and may not optimize very often.Default:
0.0
-
crossover_probability
Absolute probability of crossover-type genetic operation, instead of a mutation.
Default:
0.066
-
annealing
Whether to use annealing.
Default:
False
-
alpha
Initial temperature for simulated annealing (requires
annealing
to beTrue
).Default:
0.1
-
perturbation_factor
Constants are perturbed by a max factor of (perturbation_factor*T + 1). Either multiplied by this or divided by this.
Default:
0.076
-
skip_mutation_failures
Whether to skip mutation and crossover failures, rather than simply re-sampling the current member.
Default:
True
Tournament Selection¶
-
tournament_selection_n
Number of expressions to consider in each tournament.
Default:
10
-
tournament_selection_p
Probability of selecting the best expression in each tournament. The probability will decay as p*(1-p)^n for other expressions, sorted by loss.
Default:
0.86
Constant Optimization¶
-
optimizer_algorithm
Optimization scheme to use for optimizing constants. Can currently be
NelderMead
orBFGS
.Default:
"BFGS"
-
optimizer_nrestarts
Number of time to restart the constants optimization process with different initial conditions.
Default:
2
-
optimize_probability
Probability of optimizing the constants during a single iteration of the evolutionary algorithm.
Default:
0.14
-
optimizer_iterations
Number of iterations that the constants optimizer can take.
Default:
8
-
should_optimize_constants
Whether to numerically optimize constants (Nelder-Mead/Newton) at the end of each iteration.
Default:
True
Migration between Populations¶
-
fraction_replaced
How much of population to replace with migrating equations from other populations.
Default:
0.000364
-
fraction_replaced_hof
How much of population to replace with migrating equations from hall of fame.
Default:
0.035
-
migration
Whether to migrate.
Default:
True
-
hof_migration
Whether to have the hall of fame migrate.
Default:
True
-
topn
How many top individuals migrate from each population.
Default:
12
Data Preprocessing¶
-
denoise
Whether to use a Gaussian Process to denoise the data before inputting to PySR. Can help PySR fit noisy data.
Default:
False
-
select_k_features
Whether to run feature selection in Python using random forests, before passing to the symbolic regression code. None means no feature selection; an int means select that many features.
Default:
None
Stopping Criteria¶
-
max_evals
Limits the total number of evaluations of expressions to this number.
Default:
None
-
timeout_in_seconds
Make the search return early once this many seconds have passed.
Default:
None
-
early_stop_condition
Stop the search early if this loss is reached. You may also pass a string containing a Julia function which takes a loss and complexity as input, for example:
"f(loss, complexity) = (loss < 0.1) && (complexity < 10)"
.Default:
None
Performance and Parallelization¶
-
procs
Number of processes (=number of populations running).
Default:
cpu_count()
-
multithreading
Use multithreading instead of distributed backend. Using procs=0 will turn off both.
Default:
True
-
cluster_manager
For distributed computing, this sets the job queue system. Set to one of "slurm", "pbs", "lsf", "sge", "qrsh", "scyld", or "htc". If set to one of these, PySR will run in distributed mode, and use
procs
to figure out how many processes to launch.Default:
None
-
batching
Whether to compare population members on small batches during evolution. Still uses full dataset for comparing against hall of fame.
Default:
False
-
batch_size
The amount of data to use if doing batching.
Default:
50
-
precision
What precision to use for the data. By default this is
32
(float32), but you can select64
or16
as well, giving you 64 or 16 bits of floating point precision, respectively. If you pass complex data, the corresponding complex precision will be used (i.e.,64
for complex128,32
for complex64).Default:
32
-
fast_cycle
Batch over population subsamples. This is a slightly different algorithm than regularized evolution, but does cycles 15% faster. May be algorithmically less efficient.
Default:
False
-
turbo
(Experimental) Whether to use LoopVectorization.jl to speed up the search evaluation. Certain operators may not be supported. Does not support 16-bit precision floats.
Default:
False
-
enable_autodiff
Whether to create derivative versions of operators for automatic differentiation. This is only necessary if you wish to compute the gradients of an expression within a custom loss function.
Default:
False
-
random_state
Pass an int for reproducible results across multiple function calls. See :term:
Glossary <random_state>
.Default:
None
-
deterministic
Make a PySR search give the same result every run. To use this, you must turn off parallelism (with
procs
=0,multithreading
=False), and setrandom_state
to a fixed seed.Default:
False
-
warm_start
Tells fit to continue from where the last call to fit finished. If false, each call to fit will be fresh, overwriting previous results.
Default:
False
Monitoring¶
-
verbosity
What verbosity level to use. 0 means minimal print statements.
Default:
1e9
-
update_verbosity
What verbosity level to use for package updates. Will take value of
verbosity
if not given.Default:
None
-
progress
Whether to use a progress bar instead of printing to stdout.
Default:
True
Environment¶
-
temp_equation_file
Whether to put the hall of fame file in the temp directory. Deletion is then controlled with the
delete_tempfiles
parameter.Default:
False
-
tempdir
directory for the temporary files.
Default:
None
-
delete_tempfiles
Whether to delete the temporary files after finishing.
Default:
True
-
julia_project
A Julia environment location containing a Project.toml (and potentially the source code for SymbolicRegression.jl). Default gives the Python package directory, where a Project.toml file should be present from the install.
-
update
Whether to automatically update Julia packages when
fit
is called. You should make sure that PySR is up-to-date itself first, as the packaged Julia packages may not necessarily include all updated dependencies.Default:
False
-
julia_kwargs
Keyword arguments to pass to
julia.core.Julia(...)
to initialize the Julia runtime. The default, whenNone
, is to setthreads
equal toprocs
, andoptimize
to 3.Default:
None
Exporting the Results¶
-
equation_file
Where to save the files (.csv extension).
Default:
None
-
output_jax_format
Whether to create a 'jax_format' column in the output, containing jax-callable functions and the default parameters in a jax array.
Default:
False
-
output_torch_format
Whether to create a 'torch_format' column in the output, containing a torch module with trainable parameters.
Default:
False
-
extra_sympy_mappings
Provides mappings between custom
binary_operators
orunary_operators
defined in julia strings, to those same operators defined in sympy. E.G ifunary_operators=["inv(x)=1/x"]
, then for the fitted model to be export to sympy,extra_sympy_mappings
would be{"inv": lambda x: 1/x}
.Default:
None
-
extra_torch_mappings
The same as
extra_jax_mappings
but for model export to pytorch. Note that the dictionary keys should be callable pytorch expressions. For example:extra_torch_mappings={sympy.sin: torch.sin}
.Default:
None
-
extra_jax_mappings
Similar to
extra_sympy_mappings
but for model export to jax. The dictionary maps sympy functions to jax functions. For example:extra_jax_mappings={sympy.sin: "jnp.sin"}
maps thesympy.sin
function to the equivalent jax expressionjnp.sin
.Default:
None
PySRRegressor Functions¶
fit(X, y, Xresampled=None, weights=None, variable_names=None)
¶
Search for equations to fit the dataset and store them in self.equations_
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
X |
ndarray | pandas.DataFrame
|
Training data of shape (n_samples, n_features). |
required |
y |
ndarray | pandas.DataFrame
|
Target values of shape (n_samples,) or (n_samples, n_targets). Will be cast to X's dtype if necessary. |
required |
Xresampled |
ndarray | pandas.DataFrame
|
Resampled training data, of shape (n_resampled, n_features),
to generate a denoised data on. This
will be used as the training data, rather than |
None
|
weights |
ndarray | pandas.DataFrame
|
Weight array of the same shape as |
None
|
variable_names |
list[str]
|
A list of names for the variables, rather than "x0", "x1", etc.
If |
None
|
Returns:
Name | Type | Description |
---|---|---|
self |
object
|
Fitted estimator. |
Source code in pysr/sr.py
1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 |
|
predict(X, index=None)
¶
Predict y from input X using the equation chosen by model_selection
.
You may see what equation is used by printing this object. X should have the same columns as the training data.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
X |
ndarray | pandas.DataFrame
|
Training data of shape |
required |
index |
int | list[int]
|
If you want to compute the output of an expression using a
particular row of |
None
|
Returns:
Name | Type | Description |
---|---|---|
y_predicted |
ndarray of shape (n_samples, nout_)
|
Values predicted by substituting |
Raises:
Type | Description |
---|---|
ValueError
|
Raises if the |
Source code in pysr/sr.py
1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 |
|
from_file(equation_file, *, binary_operators=None, unary_operators=None, n_features_in=None, feature_names_in=None, selection_mask=None, nout=1, **pysr_kwargs)
classmethod
¶
Create a model from a saved model checkpoint or equation file.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
equation_file |
str
|
Path to a pickle file containing a saved model, or a csv file containing equations. |
required |
binary_operators |
list[str]
|
The same binary operators used when creating the model. Not needed if loading from a pickle file. |
None
|
unary_operators |
list[str]
|
The same unary operators used when creating the model. Not needed if loading from a pickle file. |
None
|
n_features_in |
int
|
Number of features passed to the model. Not needed if loading from a pickle file. |
None
|
feature_names_in |
list[str]
|
Names of the features passed to the model. Not needed if loading from a pickle file. |
None
|
selection_mask |
list[bool]
|
If using select_k_features, you must pass |
None
|
nout |
int
|
Number of outputs of the model.
Not needed if loading from a pickle file.
Default is |
1
|
**pysr_kwargs |
dict
|
Any other keyword arguments to initialize the PySRRegressor object. These will overwrite those stored in the pickle file. Not needed if loading from a pickle file. |
{}
|
Returns:
Name | Type | Description |
---|---|---|
model |
PySRRegressor
|
The model with fitted equations. |
Source code in pysr/sr.py
900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 |
|
sympy(index=None)
¶
Return sympy representation of the equation(s) chosen by model_selection
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
index |
int | list[int]
|
If you wish to select a particular equation from
|
None
|
Returns:
Name | Type | Description |
---|---|---|
best_equation |
str, list[str] of length nout_
|
SymPy representation of the best equation. |
Source code in pysr/sr.py
latex(index=None, precision=3)
¶
Return latex representation of the equation(s) chosen by model_selection
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
index |
int | list[int]
|
If you wish to select a particular equation from
|
None
|
precision |
int
|
The number of significant figures shown in the LaTeX
representation.
Default is |
3
|
Returns:
Name | Type | Description |
---|---|---|
best_equation |
str or list[str] of length nout_
|
LaTeX expression of the best equation. |
Source code in pysr/sr.py
pytorch(index=None)
¶
Return pytorch representation of the equation(s) chosen by model_selection
.
Each equation (multiple given if there are multiple outputs) is a PyTorch module
containing the parameters as trainable attributes. You can use the module like
any other PyTorch module: module(X)
, where X
is a tensor with the same
column ordering as trained with.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
index |
int | list[int]
|
If you wish to select a particular equation from
|
None
|
Returns:
Name | Type | Description |
---|---|---|
best_equation |
torch.nn.Module
|
PyTorch module representing the expression. |
Source code in pysr/sr.py
jax(index=None)
¶
Return jax representation of the equation(s) chosen by model_selection
.
Each equation (multiple given if there are multiple outputs) is a dictionary
containing {"callable": func, "parameters": params}. To call func
, pass
func(X, params). This function is differentiable using jax.grad
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
index |
int | list[int]
|
If you wish to select a particular equation from
|
None
|
Returns:
Name | Type | Description |
---|---|---|
best_equation |
dict[str, Any]
|
Dictionary of callable jax function in "callable" key, and jax array of parameters as "parameters" key. |
Source code in pysr/sr.py
latex_table(indices=None, precision=3, columns=['equation', 'complexity', 'loss', 'score'])
¶
Create a LaTeX/booktabs table for all, or some, of the equations.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
indices |
list[int] | list[list[int]]
|
If you wish to select a particular subset of equations from
|
None
|
precision |
int
|
The number of significant figures shown in the LaTeX
representations.
Default is |
3
|
columns |
list[str]
|
Which columns to include in the table.
Default is |
['equation', 'complexity', 'loss', 'score']
|
Returns:
Name | Type | Description |
---|---|---|
latex_table_str |
str
|
A string that will render a table in LaTeX of the equations. |
Source code in pysr/sr.py
refresh(checkpoint_file=None)
¶
Update self.equations_ with any new options passed.
For example, updating extra_sympy_mappings
will require a .refresh()
to update the equations.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
checkpoint_file |
str
|
Path to checkpoint hall of fame file to be loaded.
The default will use the set |
None
|