This phase III trial compares the effect of the combination of aglatimagene besadenovec and pembrolizumab versus standard of care docetaxel chemotherapy for the treatment of stage IV non-squamous, non-small cell lung cancer. Aglatimagene besadenovec is a replication-deficient adenoviral vector encoding the herpes simplex virus thymidine kinase (HSV-tk) gene. When combined with an oral prodrug (valacyclovir), injection of aglatimagene induces targeted tumor cell death and stimulates a systemic immune response. Immunotherapy with monoclonal antibodies, such as pembrolizumab, may help the body's immune system attack the cancer, and may interfere with the ability of tumor cells to grow and spread. Chemotherapy drugs work in different ways to stop the growth of tumor cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. This trial may help doctors find out if giving aglatimagene with pembrolizumab is more effective at treating patients with stage IV non-squamous, non-small cell lung cancer than standard chemotherapy.