The objective of the study was to determine possible differences in perception of quality of life (QoL) between patients with metastatic breast or prostate cancer, their partners, and the treating physician. Patients with metastatic breast cancer (n = 71), and metastatic prostate cancer (n = 29), a partner, and the physician each completed the same QoL questionnaire indicating how they perceived the patient's QoL. The European organization for research and treatment of cancer (EORTC) QLQ-C30 questionnaire was used to assess patients with breast cancer and the modified prostate cancer specific quality of life instrument (PROSQOLI) for patients with prostate cancer. There was reasonable agreement in mean scores between patients, and physicians or partners, for many domains of QoL; however, there was substantial discordance between scores when considering individual patients. For patients with metastatic breast cancer, physicians systematically underestimated overall QoL (p = 0.0002), social functioning (p = 0.001), and role functioning (p = 0.008), while partners showed better agreement. With prostate cancer physicians tended to underestimate pain, while mean scores for spouses were more concordant. There is substantial variability between ratings of QoL by physicians or partners, as compared to patient ratings. Medical decisions should be based on information about QoL provided by patients using validated methods.