Every paper omits something important.
Learning to find what's not written is a core research skill.
These details can change results by 10–20%. They matter enormously for reproduction.
How are embeddings initialised? Random normal? Xavier? Zero?
How is j sampled? Uniformly? Popularity-weighted? How many per positive?
Is lr constant? Does it decay? When does early stopping trigger?
What is d? Is it the same for users and items? Is it tuned?
How are ratings normalised? What threshold defines an implicit positive?
Which values were grid-searched? On which split — validation or test?
Rule of thumb: If you can't reproduce the exact experiment from the paper's "implementation details" section alone — details are missing. Check the appendix, then the code repository if available.
How are node embeddings z_n initialised before GNN_Indexing runs? Random? Pre-trained PLM only?
Section 3.8 trains with cross-entropy on (Y, A). How are negative candidate items selected during training?
Freebase has millions of triples. The paper says "filter out triples related to the three datasets" — what is the exact filtering criterion?
Eq. 6 uses sim(p, z_g) for re-ranking. Is this cosine similarity or dot product? This changes results.
Stated in Table 4 but Figure 4 shows performance is sensitive to p. Why 50% specifically? Grid searched on validation set?
Check the appendix — most papers move implementation details here under page pressure
Check the GitHub repo — if code is released, it is the ground truth implementation
Check cited papers — if they use another paper's method, the details may be in the original paper
Use field conventions — e.g., Xavier initialisation and Adam with lr=0.001 are defaults in most RS work
Email the authors — a polite, specific question usually gets a response. Attach your pseudocode.
Document your assumptions — if you fill in a gap yourself, write it down clearly. This is important for reproducibility of your own work.
Initialisation · Negative sampling · LR schedule · Embedding dims · Preprocessing · Hyperparameter tuning
When a repo exists, it is the true implementation. The paper is a description — often incomplete.
When you fill in a gap yourself, write it down. Your reproduction is only trustworthy if you know exactly what you assumed.
Next: M4 · L3 — Building an Implementation Plan