This is a study that will test how an experimental drug (enfortumab vedotin) combined with a kind of anticancer drug called an immune checkpoint inhibitor (CPI) affects patients with cancer of the urinary system (urothelial cancer). This type of cancer includes cancer of the bladder, renal pelvis, ureter or urethra that has spread to nearby tissues or to other areas of the body. Patients will get the drugs twice every 21-day cycle. Patients will get enfortumab vedotin plus the CPI on day 1. Patients will get enfortumab vedotin only on day 8. This study will look at the side effects of giving the two drugs with each other. A side effect is a response to a drug that is not part of the treatment effect. This study will also test if the cancer shrinks with treatment.