This phase II trial tests how well fluorouracil, oxaliplatin and leucovorin calcium (folinic acid) (FOLFOX) with botensilimab and balstilimab given before surgery (neoadjuvant) works in treating patients with rectal adenocarcinoma that has not spread to other parts of the body (localized). Currently, neoadjuvant therapy for rectal cancer includes chemotherapy and chemoradiation. Despite these aggressive treatments, only about half of patients achieve a complete clinical response. In fact, over half of rectal cancer patients go on to have surgery and often suffer post-surgery complications involving urine and bowel problems. Thus, there has been an increased focus on non-surgical treatments. Chemotherapy drugs, such as fluorouracil, oxaliplatin and leucovorin calcium, work in different ways to stop the growth of tumor cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. Immunotherapy with monoclonal antibodies, such as botensilimab and balstilimab, may help the body's immune system attack the tumor, and may interfere with the ability of tumor cells to grow and spread. Giving neoadjuvant FOLFOX with botensilimab and balstilimab may improve the rate of complete response and decrease the need for surgery and radiation therapy in patients with localized rectal adenocarcinoma.