[BUG]: PySR randomly hangs during regression on macOS

[ViRb3]

What happened?

Whenever I'm running regression, suddenly PySR would freeze with 0% CPU usage. If I press enter on my keyboard a few times, it resumes. This will happen entirely randomly, sometimes immediately, sometimes after 1h of regression. It goes away if I set input_stream='devnull', similar to #832.

Version

Python 3.13.3, PySR 1.5.9

Operating System

macOS

Package Manager

uv 0.9.14

Interface

Directly running the script via:

• Built-in Terminal app
• iTerm
• Ghostty

Extra Info

No response

[adots004]

I am experiencing a very similar issue. In my case, setting input_stream='devnull' seems to help but not resolve the issue entirely. It seems that using high number of iterations and max_size reduces the number of models that will be generated before hanging. Became an issue ~ 6months or so ago.

[ViRb3]

@adots004 does your hang go away if you press enter a few times on the terminal window? I feel like yours might be a different issue.

[adots004]

@ViRb3 Since I am executing the code in jupyterlab, no terminal is open. It's possible we have different issues.

Below is a bare-bones version of a helper function I use for generating the models. Started having problems I think when PysR updated how models are written to file but could be wrong.

`def generate_model(xy_train = None,
data_train = None,
data_err = None,
cust_loss = None,
num_models= 1,
prefix_label = None,
save_path = None,
errors = False,
n_iterations = 100,
model_num = None,
max_complexity = 15):

"""
Updated function with current variable names accepted by PySR. Previously "equation_file"--> output_directory + run_id

xy_train: indep variable array [[x0,y0], [x1,y0],...[xn,y0], [x0,y1], [x1,y1],...[xn,ym]
data_train: data array [f0, f1, ... f_[n+m]]
data_err: ^^ same but errors
cust_loss: Julia function string
num_models: Number of replicas to generate and write to file

Needed after pysr update(?)
replica_prefix label: "SOURCE_LOSS". Replica will be named "replica_prefix_label"_"ReplicaNumber".
save_path: folder to contain generated replicas (include / at end)
model_num: int, optional argument in case pysr hangs and have to restart at this model number.
"""

if errors == True:
    err = data_err
    weighting = [1/sig**2 for sig in err]

elif errors == False :
    weighting = np.ones(len(data_train))

if model_num is None:
    model_num = 0
for i in range(num_models):
    sr_model = sr.PySRRegressor(input_stream='devnull',#'stdin'
                    binary_operators=["+", "-", "*", "/"],#,"^"], 
                    #batching = batch_flag, 
                    #batch_size = batch_num,
                    #complexity_of_variables = var_complexity,
                    #constraints={"^": (-1, 1)},
                    loss_function = cust_loss,
                    maxsize = max_complexity, 
                    #bumper = True,
                    niterations = n_iterations,
                    output_directory = save_path +str(prefix_label),
                    run_id = 'Model '+str(model_num + i +1))
    sr_model.fit(xy_train, data_train,variable_names = ['x','y'])#, weights = weighting)

    del sr_model
return 

`

[MilesCranmer]

Thanks for the report. So, just thinking out loud, these issues likely come from the feature that monitors stdin during the search, here:
https://github.com/MilesCranmer/SymbolicRegression.jl/blob/eafea58565510cca5ed1b8f832ccf27fd233414e/src/SearchUtils.jl#L342-L360

It relies on Julia's Base.start_reading(stream), followed by repeated calls to this function:

"""Check if the user typed 'q' and <enter> or <ctl-c>."""
function check_for_user_quit(reader::StdinReader)::Bool
    if reader.can_read_user_input
        bytes = bytesavailable(reader.stream)
        if bytes > 0
            # Read:
            data = read(reader.stream, bytes)
            control_c = 0x03
            quit = 0x71
            if length(data) > 1 && (data[end] == control_c || data[end - 1] == quit)
                return true
            end
        end
    end
    return false
end

It seems pretty harmless but I don't really know how start_reading works, it seems like it does cause some issues. Here's the Julia implementation:

function start_reading(stream::LibuvStream)
    iolock_begin()
    if stream.status == StatusOpen
        if !isreadable(stream)
            error("tried to read a stream that is not readable")
        end
        # libuv may call the alloc callback immediately
        # for a TTY on Windows, so ensure the status is set first
        stream.status = StatusActive
        ret = ccall(:uv_read_start, Cint, (Ptr{Cvoid}, Ptr{Cvoid}, Ptr{Cvoid}),
                    stream, @cfunction(uv_alloc_buf, Cvoid, (Ptr{Cvoid}, Csize_t, Ptr{Cvoid})),
                    @cfunction(uv_readcb, Cvoid, (Ptr{Cvoid}, Cssize_t, Ptr{Cvoid})))
    elseif stream.status == StatusPaused
        stream.status = StatusActive
        ret = Int32(0)
    elseif stream.status == StatusActive
        ret = Int32(0)
    else
        ret = Int32(-1)
    end
    iolock_end()
    return ret
end

Maybe this uv_read_start is interacting with something in a bad way? Not sure.

But I would fully expect input_stream=devnull to make the issue go away. If it doesn't then it seems like it's something else at play.

[MilesCranmer]

I also wonder if there's a safer version of

data = read(reader.stream, bytes)

Like... maybe this is a race condition on stdin or something. And there's another process in Julia that reads some of the bytes in between the calls.

That would actually explain why pusshing buttons on the keyboard fixes it - because then read can't get the bytes it asked for.

So maybe we just need a more robust read that exits early if the remaining bytes aren't there.