FileNotFoundError [Errno 2] No such file or directory: partials0.lhe

What is the the output of the command "ulimit -l"? Such that I can try to reproduce the issue.

One workaround is to run the command "ulimit -l unlimited" and then it should work.
(I have also tested with ulimit -l 256 and it when trough too)

Cheers,

Olivier

The output of ulimit -l is 64, After set ulimit -l 60, I use the generate events -f and get the same error.

I actually made a mistake, the relevant one here is "ulimit -n"
Sorry for the mistake.

Thank you, but it seems doesn't work after I use ulimit -n 256. I get the same error.

Then it should be python specific since I do not reproduce it locally.
What is your python version?

One workaround is to move back to the LTS version (2.9.19) which should not have that issue.

Cheers,

Olivier

Thanks for your patience,

After many try I have succeed to reproduce your error, the issue is not really the ulimit but rather the large number of thread available making the (new) multi-threading method for combining event to fail due to lack of work available.

Now that I'm able to reproduce this, I should be able to produce a patch quickly,

Cheers,

Olivier

And here is the patch:

https://github.com/mg5amcnlo/mg5amcnlo/commit/6adb6eb6274f54382f6596c68c0f72e3398a9e21

Cheers,

Olivier

Thank you very much, Olivier. This patch is very useful. But, I think there are some other problem, after I use this patch(https://github.com/mg5amcnlo/mg5amcnlo/blob/6adb6eb6274f54382f6596c68c0f72e3398a9e21/madgraph/interface/madevent_interface.py), the error is coming :

By the way, I replace the file wz/bin/internal/madevent_interface.py with the patch file.

Command "generate_events -f" interrupted with error:
Exception : program /home/wsy/anaconda3/bin/python3 /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/madevent_interface.py combine_events_partial /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/Events/run_10/partials101.lhe.gz /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/Events/run_10/run_10_tag_1_banner.txt 5.12634529e-05 /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/SubProcesses/P3_qq_wpzqq_wp_lvl_z_ll/G56.025 /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/SubProcesses/P3_qq_wpzqq_wp_lvl_z_ll/G56.026 launch ends with non zero status: 1. Stop all computation
Please report this bug on https://bugs.launchpad.net/mg5amcnlo
More information is found in '/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/run_10_tag_1_debug.log'.
Please attach this file to your report.
This file's content:
#************************************************************
#* MadGraph5_aMC@NLO/MadEvent *
#* *
#* * * *
#* * * * * *
#* * * * * 5 * * * * *
#* * * * * *
#* * * *
#* *
#* *
#* VERSION 3.5.4 *
#* *
#* The MadGraph5_aMC@NLO Development Team - Find us at *
#* https://server06.fynu.ucl.ac.be/projects/madgraph *
#* *
#************************************************************
#* *
#* Command File for MadEvent *
#* *
#* run as ./bin/madevent.py filename *
#* *
#************************************************************
generate_events -f
Traceback (most recent call last):
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/extended_cmd.py", line 1546, in onecmd
    return self.onecmd_orig(line, **opt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/extended_cmd.py", line 1495, in onecmd_orig
    return func(arg, **opt)
           ^^^^^^^^^^^^^^^^
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/madevent_interface.py", line 2402, in do_generate_events
    self.run_generate_events(switch_mode, args)
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/common_run_interface.py", line 7771, in new_fct
    original_fct(obj, *args, **opts)
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/madevent_interface.py", line 2621, in run_generate_events
    self.exec_cmd('combine_events', postcmd=False,printcmd=False)
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/extended_cmd.py", line 1575, in exec_cmd
    stop = Cmd.onecmd_orig(current_interface, line, **opt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/extended_cmd.py", line 1495, in onecmd_orig
    return func(arg, **opt)
           ^^^^^^^^^^^^^^^^
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/madevent_interface.py", line 3779, in do_combine_events
    mycluster.wait(self.me_dir, update_status)
  File "/home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/cluster.py", line 847, in wait
    raise Exception(self.fail_msg)
Exception: program /home/wsy/anaconda3/bin/python3 /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/bin/internal/madevent_interface.py combine_events_partial /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/Events/run_10/partials101.lhe.gz /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/Events/run_10/run_10_tag_1_banner.txt 5.12634529e-05 /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/SubProcesses/P3_qq_wpzqq_wp_lvl_z_ll/G56.025 /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/SM/ss/wz/SubProcesses/P3_qq_wpzqq_wp_lvl_z_ll/G56.026 launch ends with non zero status: 1. Stop all computation
                              Run Options
                              -----------
               stdout_level : None

                         MadEvent Options
                         ----------------
     automatic_html_opening : False (user set)
        notification_center : True
                   run_mode : 2
              cluster_queue : None
               cluster_time : None
               cluster_size : 100
             cluster_memory : None
                    nb_core : 56 (user set)
          cluster_temp_path : None

                      Configuration Options
                      ---------------------
               pythia8_path : /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/HEPTools/pythia8 (user set)
                  hwpp_path : None (user set)
                thepeg_path : None (user set)
                 hepmc_path : None (user set)
           madanalysis_path : None (user set)
          madanalysis5_path : None (user set)
            pythia-pgs_path : None (user set)
                    td_path : None (user set)
               delphes_path : /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/Delphes (user set)
        exrootanalysis_path : None (user set)
               syscalc_path : None (user set)
                 rivet_path : None
                  yoda_path : None
                     lhapdf : /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/HEPTools/lhapdf6_py3/bin/lhapdf-config (user set)
                 lhapdf_py2 : None
                 lhapdf_py3 : /home/wsy/EasyScan_HEP-master/MG5_aMC_v3_5_4/HEPTools/lhapdf6_py3/bin/lhapdf-config (user set)
                    timeout : 0 (user set)
              f2py_compiler : None
          f2py_compiler_py2 : None
          f2py_compiler_py3 : None
                web_browser : None
                 eps_viewer : None
                text_editor : None
           fortran_compiler : None
               cpp_compiler : None
                auto_update : 0 (user set)
               cluster_type : condor
      cluster_status_update : (600, 30)
           cluster_nb_retry : 1
         cluster_local_path : None
         cluster_retry_wait : 300
#************************************************************
#* MadGraph5_aMC@NLO *
#* *
#* * * *
#* * * * * *
#* * * * * 5 * * * * *
#* * * * * *
#* * * *
#* *
#* *
#* VERSION 3.5.4 2024-04-05 *
#* *
#* The MadGraph5_aMC@NLO Development Team - Find us at *
#* https://server06.fynu.ucl.ac.be/projects/madgraph *
#* *
#************************************************************
#* *
#* Command File for MadGraph5_aMC@NLO *
#* *
#* run as ./bin/mg5_aMC filename *
#* *
#************************************************************
set group_subprocesses Auto
set ignore_six_quark_processes False
set low_mem_multicore_nlo_generation False
set complex_mass_scheme False
set include_lepton_initiated_processes False
set gauge unitary
set loop_optimized_output True
set loop_color_flows False
set max_npoint_for_channel 0
set default_unset_couplings 99
set max_t_for_channel 99
set zerowidth_tchannel True
set nlo_mixed_expansion True
import model sm
define p = g u c d s u~ c~ d~ s~
define j = g u c d s u~ c~ d~ s~
define l+ = e+ mu+
define l- = e- mu-
define vl = ve vm vt
define vl~ = ve~ vm~ vt~
generate p p > w+ z , (w+ > e+ ve) , (z > e+ e-)
add process p p > w+ z j, (w+ > e+ ve) , (z > e+ e-)
add process p p > w+ z j j, (w+ > e+ ve) , (z > e+ e-)
output ./SM/ss/wz
######################################################################
## PARAM_CARD AUTOMATICALY GENERATED BY MG5 ####
######################################################################
###################################
## INFORMATION FOR MASS
###################################
BLOCK MASS #
      5 4.700000e+00 # mb
      6 1.730000e+02 # mt
      15 1.777000e+00 # mta
      23 9.118800e+01 # mz
      25 1.250000e+02 # mh
      1 0.000000e+00 # d : 0.0
      2 0.000000e+00 # u : 0.0
      3 0.000000e+00 # s : 0.0
      4 0.000000e+00 # c : 0.0
      11 0.000000e+00 # e- : 0.0
      12 0.000000e+00 # ve : 0.0
      13 0.000000e+00 # mu- : 0.0
      14 0.000000e+00 # vm : 0.0
      16 0.000000e+00 # vt : 0.0
      21 0.000000e+00 # g : 0.0
      22 0.000000e+00 # a : 0.0
      24 8.041900e+01 # w+ : cmath.sqrt(mz__exp__2/2. + cmath.sqrt(mz__exp__4/4. - (aew*cmath.pi*mz__exp__2)/(gf*sqrt__2)))
###################################
## INFORMATION FOR SMINPUTS
###################################
BLOCK SMINPUTS #
      1 1.325070e+02 # aewm1
      2 1.166390e-05 # gf
      3 1.300000e-01 # as (note that parameter not used if you use a pdf set)
###################################
## INFORMATION FOR YUKAWA
###################################
BLOCK YUKAWA #
      5 4.700000e+00 # ymb
      6 1.730000e+02 # ymt
      15 1.777000e+00 # ymtau
###################################
## INFORMATION FOR DECAY
###################################
DECAY 6 1.491500e+00 # wt
DECAY 23 2.441404e+00 # wz
DECAY 24 2.047600e+00 # ww
DECAY 25 6.382339e-03 # wh
DECAY 1 0.000000e+00 # d : 0.0
DECAY 2 0.000000e+00 # u : 0.0
DECAY 3 0.000000e+00 # s : 0.0
DECAY 4 0.000000e+00 # c : 0.0
DECAY 5 0.000000e+00 # b : 0.0
DECAY 11 0.000000e+00 # e- : 0.0
DECAY 12 0.000000e+00 # ve : 0.0
DECAY 13 0.000000e+00 # mu- : 0.0
DECAY 14 0.000000e+00 # vm : 0.0
DECAY 15 0.000000e+00 # ta- : 0.0
DECAY 16 0.000000e+00 # vt : 0.0
DECAY 21 0.000000e+00 # g : 0.0
DECAY 22 0.000000e+00 # a : 0.0

#*********************************************************************
# MadGraph5_aMC@NLO *
# *
# run_card.dat MadEvent *
# *
# This file is used to set the parameters of the run. *
# *
# Some notation/conventions: *
# *
# Lines starting with a '# ' are info or comments *
# *
# mind the format: value = variable ! comment *
# *
# To display more options, you can type the command: *
# update to_full *
#*********************************************************************
#
#*********************************************************************
# Tag name for the run (one word) *
#*********************************************************************
  tag_1 = run_tag ! name of the run
#*********************************************************************
# Number of events and rnd seed *
# Warning: Do not generate more than 1M events in a single run *
#*********************************************************************
  10000 = nevents ! Number of unweighted events requested
  0 = iseed ! rnd seed (0=assigned automatically=default))
#*********************************************************************
# Collider type and energy *
# lpp: 0=No PDF, 1=proton, -1=antiproton, *
# 2=elastic photon of proton/ion beam *
# +/-3=PDF of electron/positron beam *
# +/-4=PDF of muon/antimuon beam *
#*********************************************************************
     1 = lpp1 ! beam 1 type
     1 = lpp2 ! beam 2 type
     6500.0 = ebeam1 ! beam 1 total energy in GeV
     6500.0 = ebeam2 ! beam 2 total energy in GeV
# To see polarised beam options: type "update beam_pol"

#*********************************************************************
# PDF CHOICE: this automatically fixes alpha_s and its evol. *
# pdlabel: lhapdf=LHAPDF (installation needed) [1412.7420] *
# iww=Improved Weizsaecker-Williams Approx.[hep-ph/9310350] *
# eva=Effective W/Z/A Approx. [2111.02442] *
# edff=EDFF in gamma-UPC [eq.(11) in 2207.03012] *
# chff=ChFF in gamma-UPC [eq.(13) in 2207.03012] *
# none=No PDF, same as lhapdf with lppx=0 *
#*********************************************************************
     nn23lo1 = pdlabel ! PDF set
     230000 = lhaid ! if pdlabel=lhapdf, this is the lhapdf number
# To see heavy ion options: type "update ion_pdf"
#*********************************************************************
# Renormalization and factorization scales *
#*********************************************************************
 False = fixed_ren_scale ! if .true. use fixed ren scale
     False = fixed_fac_scale ! if .true. use fixed fac scale
 91.188 = scale ! fixed ren scale
 91.188 = dsqrt_q2fact1 ! fixed fact scale for pdf1
 91.188 = dsqrt_q2fact2 ! fixed fact scale for pdf2
 -1 = dynamical_scale_choice ! Choose one of the preselected dynamical choices
 1.0 = scalefact ! scale factor for event-by-event scales

#*********************************************************************
# Type and output format
#*********************************************************************
  False = gridpack !True = setting up the grid pack
  -1.0 = time_of_flight ! threshold (in mm) below which the invariant livetime is not written (-1 means not written)
  average = event_norm ! average/sum. Normalization of the weight in the LHEF
#*********************************************************************
# Matching parameter (MLM only)
#*********************************************************************
 1 = ickkw ! 0 no matching, 1 MLM
 1.0 = alpsfact ! scale factor for QCD emission vx
 False = chcluster ! cluster only according to channel diag
 4 = asrwgtflavor ! highest quark flavor for a_s reweight
 True = auto_ptj_mjj ! Automatic setting of ptj and mjj if xqcut >0
                                   ! (turn off for VBF and single top processes)
 100.0 = xqcut ! minimum kt jet measure between partons

#***********************************************************************
# Turn on either the ktdurham or ptlund cut to activate *
# CKKW(L) merging with Pythia8 [arXiv:1410.3012, arXiv:1109.4829] *
#***********************************************************************
 -1.0 = ktdurham
 0.4 = dparameter
 -1.0 = ptlund
 1, 2, 3, 4, 5, 6, 21 = pdgs_for_merging_cut ! PDGs for two cuts above

#*********************************************************************
#
#*********************************************************************
# Phase-Space Optimization strategy (basic options)
#*********************************************************************
   0 = nhel ! using helicities importance sampling or not.
                             ! 0: sum over helicity, 1: importance sampling
   1 = sde_strategy ! default integration strategy (hep-ph/2021.00773)
                             ! 1 is old strategy (using amp square)
        ! 2 is new strategy (using only the denominator)
# To see advanced option for Phase-Space optimization: type "update psoptim"
#*********************************************************************
# Customization (custom cuts/scale/bias/...) *
# list of files containing fortran function that overwrite default *
#*********************************************************************
  = custom_fcts ! List of files containing user hook function
#*******************************
# Parton level cuts definition *
#*******************************
  0.0 = dsqrt_shat ! minimal shat for full process
#
#
#*********************************************************************
# BW cutoff (M+/-bwcutoff*Gamma) ! Define on/off-shell for "$" and decay
#*********************************************************************
  15.0 = bwcutoff ! (M+/-bwcutoff*Gamma)
 #*********************************************************************
 # Apply pt/E/eta/dr/mij/kt_durham cuts on decay products or not
 # (note that etmiss/ptll/ptheavy/ht/sorted cuts always apply)
 #*********************************************************************
   True = cut_decays ! Cut decay products
#*********************************************************************
# Standard Cuts *
#*********************************************************************
# Minimum and maximum pt's (for max, -1 means no cut) *
#*********************************************************************
 100.0 = ptj ! minimum pt for the jets
 25.0 = ptl ! minimum pt for the charged leptons
 0.0 = misset ! minimum missing Et (sum of neutrino's momenta)
 -1.0 = ptjmax ! maximum pt for the jets
 -1.0 = ptlmax ! maximum pt for the charged leptons
 -1.0 = missetmax ! maximum missing Et (sum of neutrino's momenta)
 {} = pt_min_pdg ! pt cut for other particles (use pdg code). Applied on particle and anti-particle
 {} = pt_max_pdg ! pt cut for other particles (syntax e.g. {6: 100, 25: 50})
#
# For display option for energy cut in the partonic center of mass frame type 'update ecut'
#
#*********************************************************************
# Maximum and minimum absolute rapidity (for max, -1 means no cut) *
#*********************************************************************
 4.5 = etaj ! max rap for the jets
 2.5 = etal ! max rap for the charged leptons
 0.0 = etalmin ! main rap for the charged leptons
 {} = eta_min_pdg ! rap cut for other particles (use pdg code). Applied on particle and anti-particle
 {} = eta_max_pdg ! rap cut for other particles (syntax e.g. {6: 2.5, 23: 5})
#*********************************************************************
# Minimum and maximum DeltaR distance *
#*********************************************************************
 0.0 = drjj ! min distance between jets
 0.4 = drll ! min distance between leptons
 0.0 = drjl ! min distance between jet and lepton
 -1.0 = drjjmax ! max distance between jets
 -1.0 = drllmax ! max distance between leptons
 -1.0 = drjlmax ! max distance between jet and lepton
#*********************************************************************
# Minimum and maximum invariant mass for pairs *
#*********************************************************************
 0.0 = mmjj ! min invariant mass of a jet pair
 0.0 = mmll ! min invariant mass of l+l- (same flavour) lepton pair
 -1.0 = mmjjmax ! max invariant mass of a jet pair
 -1.0 = mmllmax ! max invariant mass of l+l- (same flavour) lepton pair
 {} = mxx_min_pdg ! min invariant mass of a pair of particles X/X~ (e.g. {6:250})
 {'default': False} = mxx_only_part_antipart ! if True the invariant mass is applied only
                       ! to pairs of particle/antiparticle and not to pairs of the same pdg codes.
 #*********************************************************************
 # Minimum and maximum invariant mass for all letpons *
 #*********************************************************************
 0.0 = mmnl ! min invariant mass for all letpons (l+- and vl)
 -1.0 = mmnlmax ! max invariant mass for all letpons (l+- and vl)
 #*********************************************************************
 # Minimum and maximum pt for 4-momenta sum of leptons / neutrino *
 # for pair of lepton includes only same flavor, opposite charge
 #*********************************************************************
 0.0 = ptllmin ! Minimum pt for 4-momenta sum of leptons(l and vl)
 -1.0 = ptllmax ! Maximum pt for 4-momenta sum of leptons(l and vl)
#*********************************************************************
# Inclusive cuts *
#*********************************************************************
 0.0 = xptj ! minimum pt for at least one jet
 0.0 = xptl ! minimum pt for at least one charged lepton
 #*********************************************************************
 # Control the pt's of the jets sorted by pt *
 #*********************************************************************
 0.0 = ptj1min ! minimum pt for the leading jet in pt
 0.0 = ptj2min ! minimum pt for the second jet in pt
 -1.0 = ptj1max ! maximum pt for the leading jet in pt
 -1.0 = ptj2max ! maximum pt for the second jet in pt
 0 = cutuse ! reject event if fails any (0) / all (1) jet pt cuts
 #*********************************************************************
 # Control the pt's of leptons sorted by pt *
 #*********************************************************************
 0.0 = ptl1min ! minimum pt for the leading lepton in pt
 0.0 = ptl2min ! minimum pt for the second lepton in pt
 0.0 = ptl3min ! minimum pt for the third lepton in pt
 -1.0 = ptl1max ! maximum pt for the leading lepton in pt
 -1.0 = ptl2max ! maximum pt for the second lepton in pt
 -1.0 = ptl3max ! maximum pt for the third lepton in pt
 #*********************************************************************
 # Control the Ht(k)=Sum of k leading jets *
 #*********************************************************************
 1000.0 = htjmin ! minimum jet HT=Sum(jet pt)
 -1.0 = htjmax ! maximum jet HT=Sum(jet pt)
 0.0 = ihtmin !inclusive Ht for all partons (including b)
 -1.0 = ihtmax !inclusive Ht for all partons (including b)
 #*********************************************************************
 # WBF cuts *
 #*********************************************************************
 0.0 = xetamin ! minimum rapidity for two jets in the WBF case
 0.0 = deltaeta ! minimum rapidity for two jets in the WBF case
#*********************************************************************
# maximal pdg code for quark to be considered as a light jet *
# (otherwise b cuts are applied) *
#*********************************************************************
 4 = maxjetflavor ! Maximum jet pdg code
#*********************************************************************
#
#*********************************************************************
# Store info for systematics studies *
# WARNING: Do not use for interference type of computation *
#*********************************************************************
   True = use_syst ! Enable systematics studies
#
systematics = systematics_program ! none, systematics [python], SysCalc [depreceted, C++]
['--mur=0.5,1,2', '--muf=0.5,1,2', '--pdf=errorset', '--alps=0.5,1,2'] = systematics_arguments ! see: https://cp3.irmp.ucl.ac.be/projects/madgraph/wiki/Systematics#Systematicspythonmodule

Dear Olivier:

I have not encountered this problem with the same process when I use the MG5_aMC_v2_9_17. May be this will help you.

Cheers~

If you change "only" wz/bin/internal/madevent_interface.py
then you can not run the code from "./bin/mg5_aMC"
and are force to initiate the run from the one of the script of the wz/bin directory.

Given the patch enforces the combination to be done by batch of 10 as minimum and that your command is only for a batch of two, this is likely that you use "/bin/mg5_aMC" followed by a launch command which then use the unpatched file.

Cheers,

Olivier