Fix HelicalTrackHitRelations null pointers

[normangraf]

The HelicalTrackHitRelations collection contains null pointers when written out.

This appears to have changed during the "big merge."

The condition can be reproduced with the following:

java -cp hps-distribution-5.1-SNAPSHOT-bin.jar org.hps.evio.EvioToLcio -x /org/hps/steering/recon/PhysicsRun2019_pass0_recon_evio.lcsim -r -d HPS_PhysicsRun2019-v2-FEE-Pass0 evioFile -DoutputFile=tst -e 10 -n 100

Any evioFile can be used. For instance, at SLAC:

/nfs/slac/g/hps_data2/data/physrun2019/samplePartitions/hps_010022.evio.00041

or at JLab:

/cache/hallb/hps/physrun2019/data/hps_010022/hps_010022.evio.00041

[pbutti]

A basic check switching on _debug shows that the stripmap is actually filled and the relations are being formed.

HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1168e-17, 1.0000, -0.00027100]
v=[ -0.99954, 8.2462e-06, 0.030429]
vmin=-7.500000000000005
vmax=7.500000000000005
umeas=-4.289997592374341
origin=[ -7.3854, 7.7666, 37.725]
org.hps.recon.tracking.SiTrackerHitStrip1D@2161dfa4
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, 0.99500, -0.0033074]
v=[ -0.99454, 0.099842, 0.030250]
vmin=-7.500000000000005
vmax=7.500000000000005
umeas=-4.840467228810985
origin=[ -6.3481, 8.5740, 45.291]
org.hps.recon.tracking.SiTrackerHitStrip1D@67e28340
HelicalTrackHitDriver Size of hitrelations: 2
HelicalTrackHitDriver: cross at -10.12,3.83,41.61
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1168e-17, 1.0000, -0.00027100]
v=[ -0.99954, 8.2462e-06, 0.030429]
vmin=-7.500000000000005
vmax=7.500000000000005
umeas=-4.289997592374341
origin=[ -7.3854, 7.7666, 37.725]
org.hps.recon.tracking.SiTrackerHitStrip1D@2161dfa4
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, 0.99500, -0.0033074]
v=[ -0.99454, 0.099842, 0.030250]
vmin=-7.500000000000005
vmax=7.500000000000005
umeas=-4.510467228810985
origin=[ -6.3481, 8.5740, 45.291]
org.hps.recon.tracking.SiTrackerHitStrip1D@5b8f29ed
HelicalTrackHitDriver Size of hitrelations: 4
HelicalTrackHitDriver: cross at -7.09,3.83,41.52
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, -0.99500, -0.0036624]
v=[ 0.99454, 0.099852, -0.030321]
vmin=-7.5
vmax=7.5
umeas=-3.0795262396527203
origin=[ -4.1930, -8.3523, 54.015]
org.hps.recon.tracking.SiTrackerHitStrip1D@348247ed
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-7.500000000000002
vmax=7.500000000000002
umeas=-2.219422842748627
origin=[ -4.7830, -7.7412, 61.632]
org.hps.recon.tracking.SiTrackerHitStrip1D@51826332
HelicalTrackHitDriver Size of hitrelations: 6
HelicalTrackHitDriver: cross at -0.03,-5.18,57.69
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, -0.99500, -0.0036624]
v=[ 0.99454, 0.099852, -0.030321]
vmin=-7.5
vmax=7.5
umeas=-5.750931343012589
origin=[ 10.725, -6.8545, 53.560]
org.hps.recon.tracking.SiTrackerHitStrip1D@7360e8ae
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-7.500000000000002
vmax=7.500000000000002
umeas=-6.655001481405762
origin=[ 10.210, -7.7409, 61.174]
org.hps.recon.tracking.SiTrackerHitStrip1D@5ba968cb
HelicalTrackHitDriver Size of hitrelations: 8
HelicalTrackHitDriver: cross at 12.79,-1.02,57.30
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, -0.99500, -0.0036624]
v=[ 0.99454, 0.099852, -0.030321]
vmin=-7.5
vmax=7.5
umeas=-4.5645262396527215
origin=[ -4.1930, -8.3523, 54.015]
org.hps.recon.tracking.SiTrackerHitStrip1D@19701b7f
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-7.500000000000002
vmax=7.500000000000002
umeas=-3.740001481405759
origin=[ -4.7830, -7.7412, 61.632]
org.hps.recon.tracking.SiTrackerHitStrip1D@63f407b7
HelicalTrackHitDriver Size of hitrelations: 10
HelicalTrackHitDriver: cross at -1.74,-3.75,57.74
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, -0.99500, -0.0036624]
v=[ 0.99454, 0.099852, -0.030321]
vmin=-7.5
vmax=7.5
umeas=-6.15952623965272
origin=[ 10.725, -6.8545, 53.560]
org.hps.recon.tracking.SiTrackerHitStrip1D@738bc432
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-7.500000000000002
vmax=7.500000000000002
umeas=-6.050001481405761
origin=[ 10.210, -7.7409, 61.174]
org.hps.recon.tracking.SiTrackerHitStrip1D@a538872
HelicalTrackHitDriver Size of hitrelations: 12
HelicalTrackHitDriver: cross at 2.84,-1.58,57.60
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, -0.99500, -0.0036624]
v=[ 0.99454, 0.099852, -0.030321]
vmin=-7.5
vmax=7.5
umeas=-5.334506837539465
origin=[ 12.000, -7.3360, 103.54]
org.hps.recon.tracking.SiTrackerHitStrip1D@49d9ef30
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-7.500000000000002
vmax=7.500000000000003
umeas=-5.280000472187448
origin=[ 11.487, -8.2188, 111.16]
org.hps.recon.tracking.SiTrackerHitStrip1D@42d74143
HelicalTrackHitDriver Size of hitrelations: 14
HelicalTrackHitDriver: cross at 4.55,-2.84,107.57
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.099787, -0.99500, -0.0036624]
v=[ 0.99454, 0.099852, -0.030321]
vmin=-7.5
vmax=7.500000000000001
umeas=-2.694506837539464
origin=[ -2.9182, -8.8337, 104.00]
org.hps.recon.tracking.SiTrackerHitStrip1D@54d14173
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-7.500000000000002
vmax=7.500000000000002
umeas=-1.182405817745063
origin=[ -3.5063, -8.2191, 111.62]
org.hps.recon.tracking.SiTrackerHitStrip1D@467ef3f5
HelicalTrackHitDriver Size of hitrelations: 16
HelicalTrackHitDriver: cross at -7.48,-6.80,107.94
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ -0.099787, 0.99500, 0.0036624]
v=[ -0.99454, -0.099852, 0.030321]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=16.79920197848904
origin=[ 7.0820, -21.451, 203.96]
org.hps.recon.tracking.SiTrackerHitStrip1D@3e5b7f20
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=-15.960000196817523
origin=[ 7.2906, -21.447, 211.32]
org.hps.recon.tracking.SiTrackerHitStrip1D@ee301c8
HelicalTrackHitDriver Size of hitrelations: 18
HelicalTrackHitDriver: cross at -0.18,-5.39,207.88
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ -0.099787, 0.99500, 0.0036624]
v=[ -0.99454, -0.099852, 0.030321]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=16.739201978489042
origin=[ 7.0820, -21.451, 203.96]
org.hps.recon.tracking.SiTrackerHitStrip1D@3fb3aaba
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=-15.960000196817523
origin=[ 7.2906, -21.447, 211.32]
org.hps.recon.tracking.SiTrackerHitStrip1D@ee301c8
HelicalTrackHitDriver Size of hitrelations: 20
HelicalTrackHitDriver: cross at 0.43,-5.39,207.86
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ -0.099787, 0.99500, 0.0036624]
v=[ -0.99454, -0.099852, 0.030321]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=17.609064856857298
origin=[ 7.0820, -21.451, 203.96]
org.hps.recon.tracking.SiTrackerHitStrip1D@41ea1d7f
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=-15.960000196817523
origin=[ 7.2906, -21.447, 211.32]
org.hps.recon.tracking.SiTrackerHitStrip1D@ee301c8
HelicalTrackHitDriver Size of hitrelations: 22
HelicalTrackHitDriver: cross at -8.43,-5.39,208.13
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ -0.099787, 0.99500, 0.0036624]
v=[ -0.99454, -0.099852, 0.030321]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=16.139201978489044
origin=[ 7.0820, -21.451, 203.96]
org.hps.recon.tracking.SiTrackerHitStrip1D@75d53ff9
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=-15.960000196817523
origin=[ 7.2906, -21.447, 211.32]
org.hps.recon.tracking.SiTrackerHitStrip1D@ee301c8
HelicalTrackHitDriver Size of hitrelations: 24
HelicalTrackHitDriver: cross at 6.55,-5.39,207.67
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ -0.099787, 0.99500, 0.0036624]
v=[ -0.99454, -0.099852, 0.030321]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=15.381754783888095
origin=[ 10.136, -22.971, 303.91]
org.hps.recon.tracking.SiTrackerHitStrip1D@1d48d8b9
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 6.1169e-17, -1.0000, -0.00061700]
v=[ 0.99953, 1.8840e-05, -0.030536]
vmin=-49.16500000000001
vmax=49.16500000000001
umeas=-14.940000128185638
origin=[ 10.344, -22.957, 311.28]
org.hps.recon.tracking.SiTrackerHitStrip1D@1b3b30cd
HelicalTrackHitDriver Size of hitrelations: 26
HelicalTrackHitDriver: cross at 7.07,-7.92,307.70
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ -0.049028, 0.99880, -0.00085220]
v=[ -0.99838, -0.048983, 0.029121]
vmin=-49.16500000000002
vmax=49.16500000000002
umeas=14.544793904390204
origin=[ -36.510, -29.355, 505.62]
org.hps.recon.tracking.SiTrackerHitStrip1D@56c3fbef
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.00031071, -1.0000, 0.0021581]
v=[ 0.99955, 0.00024564, -0.030086]
vmin=-49.165
vmax=49.165
umeas=-13.773328363431574
origin=[ -36.288, -26.853, 513.12]
org.hps.recon.tracking.SiTrackerHitStrip1D@25d9acb3
HelicalTrackHitDriver Size of hitrelations: 28
HelicalTrackHitDriver: cross at 2.50,-12.97,508.18
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ -0.049976, 0.99875, 0.0020133]
v=[ -0.99828, -0.050014, 0.030445]
vmin=-49.165
vmax=49.165
umeas=13.259600257317029
origin=[ -30.434, -32.371, 705.51]
org.hps.recon.tracking.SiTrackerHitStrip1D@67a64d9d
HelicalTrackHitDriver:: adding relation cross<->strips
Strip with u=[ 0.00019713, -1.0000, 0.00084083]
v=[ 0.99956, 0.00017198, -0.029810]
vmin=-49.16500000000002
vmax=49.16500000000002
umeas=-11.700001082255458
origin=[ -30.287, -29.846, 713.11]
org.hps.recon.tracking.SiTrackerHitStrip1D@8a436d9
HelicalTrackHitDriver Size of hitrelations: 30

Seems like an issue in persisting them

[normangraf]

Yes, the relations are being filled correctly (else the GBL refitter would not work) during the reconstruction. The issue arises during the persistence stage.

[pbutti]

Other relations are persisted. Maybe an issue with the class being used to persist that information?

[normangraf]

My suspicion is that the new "strixel" class used for the split-strip, thin sensors might be an issue here.

[pbutti]

The class that is used to form 1D clusters should be:
https://github.com/JeffersonLab/hps-java/blob/master/tracking/src/main/java/org/hps/recon/tracking/SiTrackerHitStrip1D.java
which extends the org.lcsim.recon.tracking.digitization.sisim.SiTrackerHitStrip1D class. It's not only used for innermost layers, but it's used for all the hits

I'm unsure of the rules for object persistency in LCIO event data format or writer class, but if this is an issue related to that, could you provide informations on what could cause the persistency problem and what is wrong with the current inheritance scheme? I'm afraid the source of the problem is not fully clear to me.

[normangraf]

It turns out that this goes back much further in time, back to PR 685 which merged in iss677.

One can produce an output LCIO file which contains a readable collection of HelicalTrackHitRelations with the following:

git checkout e4cf98c

The very next merge

git checkout 01442f7

produces corrupted HelicalTrackHitRelations.

Here is what changed:

git diff e4cf98c 01442f7 --name-status

A detector-data/detectors/HPS-PhysicsRun2019-v1-4pt5/.HPS-PhysicsRun2019-v1-4pt5.lcdd.swp
M detector-model/src/main/java/org/lcsim/detector/tracker/silicon/HpsThinSiSensor.java
A detector-model/src/main/java/org/lcsim/detector/tracker/silicon/SiStriplets.java
M ecal-readout-sim/src/main/java/org/hps/readout/DigitizationReadoutDriver.java
M ecal-readout-sim/src/main/java/org/hps/readout/RawConverterReadoutDriver.java
M ecal-readout-sim/src/main/java/org/hps/readout/hodoscope/HodoscopeDigitizationReadoutDriver.java
A ecal-readout-sim/src/main/java/org/hps/readout/hodoscope/HodoscopePatternReadoutDriver.java
M ecal-readout-sim/src/main/java/org/hps/readout/rawconverter/AbstractBaseRawConverter.java
A ecal-readout-sim/src/main/java/org/hps/readout/trigger2019/PairsTrigger2019ReadoutDriver.java
A ecal-readout-sim/src/main/java/org/hps/readout/trigger2019/SinglesTrigger2019ReadoutDriver.java
A record-util/src/main/java/org/hps/readout/util/HodoscopePattern.java
A record-util/src/main/java/org/hps/record/triggerbank/TriggerModule2019.java
A steering-files/src/main/resources/org/hps/steering/readout/PhysicsRun2019TrigPairs.lcsim
A steering-files/src/main/resources/org/hps/steering/readout/PhysicsRun2019TrigPulse.lcsim
A steering-files/src/main/resources/org/hps/steering/readout/PhysicsRun2019TrigSingles.lcsim
A steering-files/src/main/resources/org/hps/steering/recon/PhysicsRun2019MCRecon.lcsim
A steering-files/src/main/resources/org/hps/steering/recon/PhysicsRun2019Recon.lcsim
D steering-files/src/main/resources/org/hps/steering/recon/PhysicsRun2019_testRecon.lcsim
M tracking/src/main/java/org/hps/recon/tracking/DefaultSiliconResolutionModel.java
M tracking/src/main/java/org/hps/recon/tracking/HelicalTrackHitDriver.java
A tracking/src/main/java/org/hps/recon/tracking/SiTrackerHitStrip1D.java
M tracking/src/main/java/org/hps/recon/tracking/StripMaker.java
M tracking/src/main/java/org/hps/recon/tracking/gbl/MakeGblTracks.java
M tracking/src/main/java/org/hps/recon/tracking/kalman/KalmanInterface.java
M tracking/src/main/java/org/hps/recon/tracking/kalman/KalmanPatRecDriver.java
M tracking/src/main/java/org/hps/recon/tracking/kalman/KalmanPatRecHPS.java

I continue to suspect it is related to the introduction of the SiStriplets.

[normangraf]

I've committed a simple test case which can be used to debug this issue.

[JeremyMcCormick]

@normangraf This is fixed now, right?