`torch_dtype` is deprecated! Use `dtype` instead!
ground truth verified by execution for all 9 cases


Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]
Loading checkpoint shards:  33%|███▎      | 1/3 [00:01<00:03,  1.65s/it]
Loading checkpoint shards:  67%|██████▋   | 2/3 [00:03<00:01,  1.84s/it]
Loading checkpoint shards: 100%|██████████| 3/3 [00:03<00:00,  1.08s/it]
Loading checkpoint shards: 100%|██████████| 3/3 [00:03<00:00,  1.27s/it]
The module name  (originally ) is not a valid Python identifier. Please rename the original module to avoid import issues.
model=google/gemma-2-2b (local snapshot c5ebcd40) dtype=torch.bfloat16

=== ctrl-no-reassign ===
    control: no reassignment; ',' = continues to 3 (correct), ']' = stops
    last 5 tokens: ['[', '1', ',', ' ', '2']
   98.44%  ','
    1.40%  ']'
    0.01%  ' ,'
    0.00%  '.'
    0.00%  '3'
    0.00%  '],'
    0.00%  ' '
    0.00%  ',]'
    0.00%  ' ...'
    0.00%  '];'
  -> P(',') =  98.44%   [correct (continues , 3])]
  -> P(']') =   1.40%   [wrong (stops early)]

=== ctrl-plain-five ===
    control at the length the test0 cases use, so ']' there cannot be a taste for short lists
    last 5 tokens: ['[', '1', ',', ' ', '2']
   99.61%  ','
    0.41%  ']'
    0.00%  ' ,'
    0.00%  '.'
    0.00%  '3'
    0.00%  '],'
    0.00%  '...]'
    0.00%  ' '
    0.00%  '\n'
    0.00%  ' ...'
  -> P(',') =  99.61%   [correct (continues , 3, 4, 5])]
  -> P(']') =   0.41%   [wrong (stops early)]

=== ctrl-filter-no-reassign ===
    control: the filter is present but vacuous without reassignment; still [1..5]
    last 5 tokens: ['[', '1', ',', ' ', '2']
   89.45%  ','
   10.64%  ']'
    0.01%  ' ,'
    0.00%  ',]'
    0.00%  '.'
    0.00%  '],'
    0.00%  '...]'
    0.00%  ']#'
    0.00%  ' ...'
    0.00%  ' ...]'
  -> P(',') =  89.45%   [correct (the filter drops nothing)]
  -> P(']') =  10.64%   [wrong (stops early)]

=== genexp-first-element ===
    genexp form: iterates OLD [1..5] (eager), filters by NEW [0,1,2]; first char after '['
    last 5 tokens: ['(', 'g', '))', '\n', '[']
   55.08%  '0'
   37.70%  '1'
    5.10%  '2'
    0.89%  '5'
    0.54%  '4'
    0.37%  '3'
    0.08%  ' '
    0.08%  'False'
    0.08%  'True'
    0.06%  'None'
  -> P('1') =  37.70%   [correct (iterable was captured eagerly)]
  -> P('0') =  55.08%   [wrong (would mean the iterable is lazy too)]

=== test2-first-element ===
    same program desugared; iterates OLD [1..5], filters by NEW [0,1,2] -> [1, 2]
    last 5 tokens: ['(', 'g', '))', '\n', '[']
   40.43%  '0'
   40.43%  '1'
   13.09%  '2'
    2.28%  '4'
    1.78%  '3'
    1.56%  '5'
    0.19%  ' '
    0.04%  'None'
    0.04%  '\n'
    0.02%  'False'
  -> P('1') =  40.43%   [correct (iterates old lst)]
  -> P('0') =  40.43%   [wrong (iterates new lst)]

=== genexp-where-stop ===
    genexp form: correct output is [1, 2]; ']' = filter uses NEW lst, ',' = filter uses OLD
    last 5 tokens: ['[', '1', ',', ' ', '2']
   53.12%  ']'
   46.88%  ','
    0.01%  ',]'
    0.01%  '],'
    0.00%  ' ,'
    0.00%  ']#'
    0.00%  '.'
    0.00%  '];'
    0.00%  ' '
    0.00%  '\n'
  -> P(']') =  53.12%   [correct (filter resolved lazily = new lst)]
  -> P(',') =  46.88%   [wrong (filter = old lst, i.e. nothing dropped)]

=== test2-where-stop ===
    same program desugared; correct output is [1, 2]; ']' = filter uses NEW lst
    last 5 tokens: ['[', '1', ',', ' ', '2']
   73.05%  ']'
   26.95%  ','
    0.02%  ',]'
    0.01%  '],'
    0.00%  '];'
    0.00%  ' '
    0.00%  ' ,'
    0.00%  ' ]'
    0.00%  ']#'
    0.00%  '\n'
  -> P(']') =  73.05%   [correct (filter = new lst)]
  -> P(',') =  26.95%   [wrong (filter = old lst)]

=== test1-third-pair ===
    correct 3rd pair is (2, 1); '(1, 3)' would mean the outer loop used old lst lazily... vice versa
    last 5 tokens: [',', ' ', '2', '),', ' (']
   99.22%  '2'
    0.59%  '1'
    0.01%  '3'
    0.00%  ' '
    0.00%  '\n'
    0.00%  'a'
    0.00%  '4'
    0.00%  '0'
    0.00%  '5'
    0.00%  '...,'
  -> P('2') =  99.22%   [correct (inner b exhausted at 2)]
  -> P('1') =   0.59%   [wrong ((1, 3): inner = old lst)]

=== test1-does-3-appear ===
    after (2, 2): ',' = continues to (3, ...) (outer eager, correct), ']' = outer used new lst
    last 5 tokens: ['2', ',', ' ', '2', ')']
   53.91%  '\n'
   14.45%  ' ]'
   14.45%  '\n\n'
    6.84%  '\n\n\n'
    3.44%  '...]'
    1.43%  '<eos>'
    0.58%  ' '
    0.41%  ' ...]'
    0.38%  '()]'
    0.37%  ' ...'
  -> P(',') =   0.00%   [correct (outer eager: (3, 1) follows)]
  -> P(']') =   0.00%   [wrong (outer = new lst)]

=== summary (first candidate listed = the correct one) ===
  ctrl-no-reassign         P(',')= 98.44%  P(']')=  1.40%   margin=97.03%   ok
  ctrl-plain-five          P(',')= 99.61%  P(']')=  0.41%   margin=99.20%   ok
  ctrl-filter-no-reassign  P(',')= 89.45%  P(']')= 10.64%   margin=78.81%   ok
  genexp-first-element     P('1')= 37.70%  P('0')= 55.08%   margin=17.38%   MISS
  test2-first-element      P('1')= 40.43%  P('0')= 40.43%   margin= 0.00%   tie
  genexp-where-stop        P(']')= 53.12%  P(',')= 46.88%   margin= 6.25%   ok
  test2-where-stop         P(']')= 73.05%  P(',')= 26.95%   margin=46.09%   ok
  test1-third-pair         P('2')= 99.22%  P('1')=  0.59%   margin=98.63%   ok
  test1-does-3-appear      P(',')=  0.00%  P(']')=  0.00%   margin= 0.00%   noise
  'margin' is the gap between the two candidates. It is reported, not
  thresholded: no cutoff below which a win stops counting is claimed here.

=== genexp vs its desugaring (same program, must match) ===
  which list is iterated     inconclusive (test2-first-element = noise or an exact tie)
  which list the filter uses agree on ']'
  Agreement is necessary, not sufficient: two matching wrong answers
  still agree. A disagreement rules out the semantics outright --
  PEP 289 defines the genexp to BE the desugared function.