Cancer Registry Survival Data for Metrics of Continuous Quality Improvement and Quality Assurance

A B S T R A C T

Purpose: The purpose of this article is to illustrate how cancer registry data can be used to address questions of quality assurance and continuous quality improvement, and to generate contemporary cohorts of patients for retrospective studies. The history and purpose of hospital cancer registries is reviewed and examples of use of registry data provided.
Methods: Cancer Registry information, manuals, and definitions were reviewed. The 25-year experience of the lead author in collaborating with registrars and using the cancer registry of a large community hospital in southern California is described. The strengths and weaknesses of such data are discussed. Examples of completed studies are provided to illustrate how such data was organized, analyzed, and presented to physicians and administrators, and for peer reviewed publications.
Results: The strengths of such data are the large numbers of patients, validity of date of diagnosis, histologic diagnosis, general stage, and date of death. The major limitations of the data are due to incomplete reporting of specific treatment regimens, especially after the initial 4-month period of management. The quality assurance and quality improvement studies generated proved to be of great interest to local physicians and administrators. Several such studies were used in peer-reviewed publications. The interest and job satisfaction of registrars, and data quality all improved when registry data was being used and reported locally rather than merely being submitted to a national data repository.
Conclusion: In the absence of comprehensive integrated medical care data bases, high-quality cancer registries can be used to address local issues related to quality improvement and quality assurance and provide data for peer-reviewed publications.

Keywords

Cancer registries, tumor registries, outcomes, survival, quality

Introduction

I Survival as A Metric for High Quality Cancer Care

There are increasing demands for transparency in health care, including making cost and outcome data more readily available to consumers and payers in order to establish the quality of care delivered [1-3]. Some have suggested that we emphasize the cost/benefit (i.e. value) of health care services based on the ratio of medical care costs incurred to a metric of desired clinical outcome [4]. Invasive cancer is the number one killer of Americans under 85 years of age; so, it is not surprising that the cost and value of cancer care is often targeted in discussions of health care reform. Survival is almost always the primary objective endpoint or at least a secondary endpoint of randomized clinical trials. Survival is the major focus of most cancer patients and is a good indicator of cancer-care quality because it is influenced by early detection procedures, cancer-directed treatment by various specialists, and management of co-morbid medical conditions by various practitioners. Survival can be easily and reliably measured because the date of diagnosis of invasive cancer is documented in pathology reports and cancer registry abstracts, and date of death is a well-documented vital statistic. Despite these advantages, other than the reporting of national survival data by the Surveillance Epidemiologic End-Result (SEER) program, survival is seldom used as an indicator of continuous quality improvement (CQI) or as a quality assurance (QA) indicator of cancer care [5-8]. Until efficient, integrated, electronic medical systems are well-established, existing cancer registry data can be used to assess the quality of cancer care for any hospital or hospital system that manages large numbers of cancer patients.

II History of Cancer Registries

Hospital-based cancer registries were first established almost a century ago in order to collect cancer-related information, including social demographics, histology, stage, treatment, date of diagnosis, and date of death [9]. In 1926 the first hospital-based registry was established at Yale-New Haven in Connecticut. Over the next decade eight additional cancer registries were established in Connecticut, as well as in the Veterans Administration Hospital in Portland, Oregon, a hospital in Decatur, Georgia, and at the Universities of Nebraska, Virginia, Michigan, and Southern California. The first state to register cancer cases was Massachusetts in 1927, followed by Connecticut in 1935. California started as a state registry in 1947 with nine hospitals.

The numbers of hospital-based registries expanded dramatically after 1956 when the American College of Surgeons began requiring tumor registries in order to be designated as a cancer program. As part of the declaration of “War on Cancer” and the National Cancer Act of 1971, the National Cancer Institute’s SEER program was established in 1973 specifically to track national cancer incidence and 5-year relative survival rates for 24 major tumor types [5]. Population-based cancer registries were created to monitor the annual frequency of new cancer cases, but in well-defined populations rather than hospitals. For many years SEER relied on data from a sample of about 10% of the population, but in 2000 this was increased to about 28%. In 1992 Congress enacted the Cancer Registries Amendment Act, which established the National Program of Cancer Registries (NPCR) under the auspices of the Centers for Disease Control and Prevention (CDC) [10]. Their objective was to collect cancer incidence and mortality data from all 50 states including data for in situ and invasive cancers for each major histologic cancer type. Cutaneous melanoma was one of these cancer types, but cutaneous squamous cell and basal cell cancers were excluded. The act also specified the need to set standards for protection of confidentiality of cancer case data as well as “a means by which confidential case data may in accordance with State law be disclosed to cancer researchers for the purposes of cancer prevention, control and research”. By 1993 cancer was considered a “reportable disease” throughout the United States.

In 1998 a related Public Law Congressional Mandate authorized the CDC to provide states funds in order to (1) improve existing registries, (2) implement new cancer registries (3) establish legislation and regulations so states could enhance the viability and visibility of registry operations, (4) create standards for data quality, including completeness, accuracy, and timeliness, (5) provide training for registry personnel, and (6) help establish a computerized reporting and data processing system. This act also expanded the tumor types for which information was to be collected to include benign brain-related tumors (including pituitary) rather than just malignant brain tumors. Highly-trained cancer registrars, also known as tumor registrars, are the personnel who collect relevant data related to the diagnosis and treatment of cancer, including the histologic type of cancer, to help determine cancer incidence, and the date of diagnosis, histologic type and stage of disease, and follow up dates to determine survival. Numerous regional and national associations of registrars have been organized over the years.

The large California Cancer Registrars Association (CCRA) was established in 1973. In 1974 the National Tumor Registrars Association (NTRA) was founded in Texas and is now called the National Cancer Registrars Association (NCRA). Its members include Certified Tumor Registrars (CTR) and other cancer registry professionals. They provide education programs and workshops for registrars, have granted exam-based certification since 1983, and have held annual conferences for more than 45 years. The population-based central registries in the United States and Canada are members of the North American Association of Central Cancer Registries (NAACR), which was founded in 1987. This group certifies registries, sets quality standards for data and for transmission of digital cancer records, and collaborates with SEER, NCI, CDC, and NPCR for annual reporting of national cancer statistics.

III Using a Hospital Cancer Registry for Continuous Quality Improvement (CQI) and Quality Assurance (QA)

Most hospitals and systems support a cancer registry in order to be in compliance with federal and state statutes, but they do not actually utilize the information contained in the cancer registry in a meaningful way. However, the data contained in these registries can be quite useful for quality-related studies including both CQI and QA. An institution’s own registry data can be used to establish historical survival rates as benchmarks for internal (intramural) comparisons over time. These are historical comparisons, and any time intervals that provide sufficient numbers of patients can be useful for this purpose. The institution’s data can also be compared to contemporary national survival rates reported by SEER as an external benchmark. In this case the same years covered in the SEER analysis must be used so that the comparison is to contemporary outcomes. Internal data can be readily compared using observed or relative 5-year survival rates and/or median survivals. Comparisons to national SEER data is mostly easily done with relative 5-year survival rates that approximate cancer-specific survival by taking into account competing causes of death based on age and gender. Herein we describe how the cancer registry data from a large community hospital and cancer center was used to assess CQI and QA for more than 25 years and used as components of multiple peer-reviewed publications.

Methods

I Setting

Hoag Hospital is a 498-bed not-for-profit community hospital located in Newport Beach, California, in Orange County, which has a population of about 3.1 million. Between 1990 and 2014 the hospital’s primary service area included about one million individuals. The hospital had supported a cancer registry since 1981, but cancer patient numbers were quite small until 1986 when additional medical oncology-trained cancer specialists joined the medical staff. In the fall of 1990, when the Patty and George Hoag comprehensive outpatient cancer center opened on the hospital campus, the registry was physically transferred out of the hospital and into the Cancer Center facility, and administrative oversight was transferred to the administrative and medical directors of the Cancer Center. The registry was upgraded with personnel additions, new computers, and new software programs. In each year during 1990-2014 Hoag physicians managed the highest volume of newly diagnosed cancer patients in Orange county, and the registry volume was one of the largest in southern California, usually third in number of newly diagnosed patients, only behind UCLA and Cedars-Sinai in Los Angeles. After the opening of the Cancer Center facility, the cancer program was continually accredited as a comprehensive community cancer program by the Commission on Cancer of the American College of Surgeons and received “outstanding” designations on multiple occasions. The cancer program was consistently ranked in the Top 100 by US News and World Report, and by Becker’s Hospital Review. During 1989 to 2014 the annual number of cases abstracted by the registry increased from about 1000 to over 3000.

II Cancer Registry Data Sets and Subsets

Information for cancer patients who were diagnosed and/or treated at Hoag Hospital and its outpatient facilities was compiled regularly by full-time registrars per standards published by the NCRA [9]. Over time registry personnel increased in number to include a head registrar and three associate-certified registrars. Data was entered and stored electronically using computer software designed for this purpose (Electronic Registry Systems, Inc., Cincinnati, OH). Each basic report had the following worksheets available for analysis after transfer into Excel® software programs: Miscellaneous Report Labels, General Summary Stage, AJCC stage, and then Survival and Relative Survival for all patients and in situ, local, regional, distant, and unknown stages of disease. Any of the variables in the Miscellaneous Report can be selected for subset analyses with the same categories and worksheets.

Cancers were classified by histologic diagnosis and characterized as “in situ” or “invasive” based on pathology reports, and each case was classified as “analytical” or “non-analytical” [9]. By definition, analytical cases were those patients who were either diagnosed at Hoag (or the offices of Hoag physicians) and/or treated in Hoag Hospital facilities or by Hoag physicians in private offices, within four months of the cancer diagnosis. Non-analytical cases encompassed patients who had received all their primary cancer care elsewhere, but subsequently had come to medical attention at Hoag Hospital, usually because of progressive disease. For purposes of outcomes comparisons, data was limited to analytical cases of invasive cancer, except for bladder cancer, because in situ bladder cancer is included in survival statistics per SEER methodology.

SEER methodology was used to define patient populations by stage and to calculate relative 5-year survival rates. Standard computerized reports were generated for the entire population of analytical invasive cancer patients and for each of the 24-specific tumor-types reported by SEER. Per SEER convention, basal cell and squamous cell carcinomas of the skin, borderline ovarian tumors, and in situ carcinomas of all sites except bladder were excluded. The head registrar transferred the data into Excel® spread sheets for analysis. For more in-depth analyses, separate reports were prepared for subsets defined by a variable of interest such as stage, age group, gender, treatment, specific histology, race/ethnicity, or combinations of these. Reports for such subsets included all the information described above.

In addition to survival for all patients, subset comparisons for survival were made using the general staging classifications of local, regional, and distant metastatic disease as defined by SEER, rather than AJCC stages that continually change over time, and therefore are not useful for historical comparisons [11, 12]. Each subset comparison included the proportion of patients with local, regional, distant and unknown stage disease, and the observed survival rate for each stage, and for all patients with the specific type of malignancy.

III Statistical Analyses

Actuarial life-table survival rates and relative 5-year survival rates were calculated using computer software (Electronic Registry Systems, Inc., Cincinnati, OH.). Data was directly transferred into Excel® for various statistical packages to generate survival curves and statistical analysis including log rank tests. Two-tailed tests of probability were used for statistical comparisons.

Survival was calculated from the date of diagnosis to the date of last follow up, or date of death (if deceased). Numbers of patients alive and numbers of deaths and censored events were recorded by month from diagnosis; therefore, the Mantel-Haenszel log-rank test was used for comparisons of survival curves. Relative survival, as reported by SEER, is the ratio of observed survival for cancer patients to the expected survival for the general population with adjustments for competing causes of mortality based on age, race, and gender, to estimate cancer mortality rate rather death from all causes [13, 14].

It should be noted that the reference year used by SEER to estimate survival of the general population for estimating relative survival has changed over time. For many years 1980 population data was used to calculate expected survival rates, but more recently 2004 population data has been used as the standard year for expected survival rates. Pearson Chi-Square tests were used for comparisons of proportions unless very small numbers necessitated use of the Fisher Exact Test. For internal comparisons, observed median survival and 5-year survival rates were typically used while for external comparisons to SEER data, relative 5-year survival was used. For specific survival-curve comparisons, SEER will provide the actual data used to generate the 5-year survival rates that are well-publicized each year and published on line as well is in the Cancer CA Journal for Clinicians [15].

Results

I Data Quality

High-quality data is critical for such analyses, otherwise there is a high potential for “garbage-in, garbage out.” The American College of Surgeons has prescribed physician review of 10% of the number of analytical cases as a way to determine registry quality, but this has significant limitations, and does not detect clerical errors that are made frequently during data entry into electronic systems. Many types of errors were identified and corrected when data sets were transferred into Excel® files in which sorting procedures made it easy to identify outlier data. Examples included missing entries, erroneous date entries, failure to exclude certain tumor types such as sarcomas that might originate in the lung but are not histologic lung cancers. Timely follow up was maintained on more than 90% of patients for five years after diagnosis, and on more than 80% for 10 years. A high follow up rate is critical to avoid underestimating death rates, and conversely, overestimating survival rates [16].

In general, the pathology diagnosis, stage, date of diagnosis and date of death were reliably reported because nearly all surgery took place in Hoag hospital or in surgical centers that provided data to the registry. Radiation therapy data was of high quality because nearly all radiation therapy was given in the Hoag Cancer Center, and all records were maintained there. However, systemic treatment details were much less reliable since most long-term patient management occurred in the outpatient treatment setting in the offices of medical oncologists. The general categories of treatment were reliable, but it was not always possible to examine the use of specific systemic treatment regimens. The following anecdotal experience exemplifies the limitations of such registries when all outpatient and inpatient data is not linked by an efficient electronic system. During an airline flight, the lead author engaged in conversation with a stewardess who advised him that she was a colon cancer patient who had undergone surgery in Hoag Hospital, and had for several years been under the care of a specific medical oncologist, after having earlier been under the care of a different medical oncologist. Based on the information she provided, several entries in her registry abstracts was found to be erroneous. All her basic information (date of diagnosis, tumor type, stage, and surgical procedure) were correct, but the medical oncologist of record was the one who had seen her in the hospital, and the chemotherapy listed was what he had dictated in his consultation report. It turned out the patient had subsequently sought care from a different medical oncologist who treated her with a different adjuvant chemotherapy regimen. The registrars had requested follow up information from her known physicians, but the surgeon was no longer seeing her, she had not received radiation therapy, and the medical oncologist of record had not seen her since she left the hospital; so, he and his staff could provide no follow up information. The patient herself was busy working; so, she had not responded to update inquiries. Based on the information she provided, registry personnel were able to contact her new medical oncologist who provided the additional information, and her follow up information was corrected and updated such that she was no longer censored as being lost to follow up shortly after surgery. Review of this patient’s abstract or hospital chart would not have detected the recorded errors related to the managing medical oncologist and the specific chemotherapy she received.

Table 1:  Miscellaneous Report Labels:  Categories and Content.

Title	Content
Site Code	Tumor type & number of cases
Total Cases	Total number of cases
Histology	Each pathology diagnosis and numbers of each and proportion of all diagnoses.  There can be numerous subtypes; so for subset analyses, best to group these together, which is facilitated by sorting alphabetically.  For instance, a lung cancer file had 48 histologies, but these could be grouped into adeno, squamous, neuroendocrine, small cell, and others. Also allows quality check to exclude histologies such as lymphoma and sarcoma that occurred in that site.
General Stage	Numbers and percentages by local, regional, distant, and unknown stage and substages of regional to include direct extentsion, lymph node involvement, or both.  This is assigned by registrars and is the staging used by SEER.
Summary AJCC Stage Group	Stages I, II, III, IV with A, B, C subsets.  Because of detail required, much higher percentage likely to be classified as unknown stage.  For instance in a lung cancer report, only 2% had an unknown general stage, but 20% had an unknown AJCC stage.  Furthermore, AJCC stage definitions have changed over time, making it hard to use for historical  comparisons.
Age at Diagnosis	Numbers and percentages by age. Easy to detect some errors.  For instance, in one lung report, 6 of 1,334 were listed as heaving a negative age. By sorting, can readily see age range.
Race	White, Black, Chinese, Japanese, Vietnamese, Korean, other Asian, Filipino, American Indian, other, and unknown.  Have to combine with ethnicity if want to do subset analysis on Hispanics and/or non-Hispanic Whites.
Accession Year	Shows number and percentage of cases by year of diagnosis and accession into the registry.
Grade	Undifferentiated/anaplastic, poorly-, moderately- , or well-, differentiated, or unknown.  Useful prognostic marker.
Class of Case	Non-analytical had already been excluded; so these were; [1] 1st DX here, Rx here, [2] 1st Dx elsewhere, Rx here, and [3] 1st DX here, Rx elsewhere. Useful to determine outmigration of patients for therapy, and referral of patients for therapy.
Dx Confirmation Code	Numbers and percentages by positive histology, positive cytology without histologic confirmation, radiographic diagnosis only, clinical diagnosis only, unknown, and others
Ethnicity	Non-Spanish, Spanish, Mexican , Cuban, etc. Can be combined with Race to create large groups such as Asian, Black, Hispanic and non-Hispanic White
Sex	Number and percentage by gender: male, female, and other
Primary Payer	HMO, PPO, Medicare, Medicaid, Uninsured, Medicare through managed care, Medicare with supplement, and others
Primary Medical Oncologist	Name and percentage of cases, including none. However, reflects MD of record at time of initial diagnosis, and not uncommon for patients to change physicians in the course of care, and sometime non-med oncs listed.
Primary Surgeon	Name and percentage of cases, including none.  Because of surgical reports at diagnosis, names listed are accurate
Rx Type	Treatments in order by number and percentage.  Needs to be sorted alphabetically to group.  Options include surgery, radiation, chemotherapy, hormonal therapy immunotherapy, and various combinations, and “other.”  Common inaccuracies in the systemic therapies listed, especially if not started until after completion of surgery and radiation and/or if treatment was given in physician’s infusion center rather than in a hospital-based facility.

II Registry Data

The first Worksheet in each Excel file was entitled “Miscellaneous Report Labels” (Table 1). This sheet included numbers of patients, numbers and proportions of specific cancer histologies (squamous cell, adenocarcinoma, small cell, etc.), method of diagnosis (histology, cytology, clinical) tumor differentiation grade (unknown, well-, moderately- or poorly-differentiated), general stage (local, regional, distant), stage per American Joint Committee on Cancer (AJCC) as assigned by managing physicians, ages at diagnosis, race, ethnicity, sex, accession year, class of case (analytical or non-analytical, and if analytical, whether diagnosed and treated at Hoag; diagnosed elsewhere but treated at Hoag; diagnosed at Hoag but all of initial treatment was delivered elsewhere; treatment type (none, surgery, radiation therapy, chemotherapy, hormonal therapy, biological or immunotherapy, and various combinations and permutations of these) [11].

Table 2: Content of survival data in Excel^® worksheets.

Categories	Content
Title  (top of page)	Description of data set or subset and years of inclusion
Interval month (Column A)	Consecutive number by months from date of diagnosis
Alive in month (Column B)	Number of patients known to be alive at the start of this month
Dead in month (Column C)	Number of patients known to have died during that month
Last seen in month (Column D)	Number of patients known to still be alive as of that month, but no subsequent follow up information
Exposed to Risk (Column E)	Computer calculation based on previous 3 columns for calculation used in actuarial survival figure
Proportion Dying (Column F)	Column C divided by Column B
Proportion Surviving (Column G)	Column B minus Column D, divided by Column B
Proportion at end of interval (Column H)	Calculation of the Kaplan-Meir statistic for overall survival (all-cause mortality)
Proportion relative survival (Column I)	Adjusted calculation of Column H to produce the relative survival statistic (adjusted for competing causes of death based on age and gender of those dying in the interval compared to expected rates for year in use.

The second worksheet was entitled “General Summary Stage,” and tabulated by each year covered in the report, the distribution of patients in each general stage: local, regional direct extension, regional lymph nodes, regional by both extension and lymph node involvement, and distant. The third worksheet showed yearly distribution of cases by AJCC stage. Unfortunately, the latter was never as complete as the general stage, because AJCC TNM classification depended on physician designation while general stage could be assigned by the registrars. The subsequent worksheets tabulated survival data as described in (Table 2), including both Kaplan-Meier actuarial data and relative survival data. Successive worksheets addressed all patients, and cohorts defined by local, regional, distant and unknown stage. Any of those variables can also define a subset cohort for a detailed analysis to understand all of the miscellaneous characteristics for patients in each stage of disease. Column H of each data set includes the actuarial data for plotting an overall survival curve. The 1-, 2-, 3-, 4, and 5-year survival rates are the figures in Column H across from months 12, 24, 36, 48, and 60. The median survival is the month after the 50% figure in Column H. The 60-month (5-year) survival proportion in Column G is the relative survival used for comparisons to SEER relative survival rates.

Table 3: Characteristics of patients diagnosed with "invasive cancer* during the successive eras 1989-1995 and 1996-2003.

Category	1989-1995 n=7,662 patients	1996-2003 n=12,977 patients	Chi Square P value
Median Age in years	65.0	65.0	---
% “White” non Hispanic	94.4	92.2	<.0001
% classified as “Asian”	2.6	3.6	<.0001
% Hispanic	2.6	3.9	<.0001
% “African-American”	0.43	0.34	.30
% Female	52.0	52.9	.21
% < age 20 years	0.43	0.49	.53
% > age 90 years	0.97	1.39	.0080
% in situ bladder	2.0	2.2	.34
% local	46.8	50.2	<.0001
% regional	21.6	20.7	.12
% distant	23.6	20.6	<.0001
% unknown general stage	6.0	6.3	.39
% Diagnosed & treated at Hoag	83.2	83.5	.58
% Diagnosed elsewhere,  treated at Hoag	15.4	15.3	.84
% Diagnosed at Hoag, treated elsewhere	1.4	1.2	.23
% Diagnosed by histopathology	91.8	92.1	.45
% Diagnosed by cytology only	5.8	5.5	.38

*includes tumors classified as “unknown stage;” excludes “in situ” except for bladder.

III Data for Comparisons to Internal and External Benchmarks

High level QA and CQI comparisons to national benchmarks focused on readily available 5-year relative survival data as reported by SEER [13, 14]. Comparisons to internal benchmarks focused on actuarial survival data in the registry, and at times relative survival as well. Basic subsets included all patients and each of 24 tumor types: lung, breast, prostate, bladder, kidney, testis, colon, rectal, esophagus, stomach, pancreas, liver, cervix, uterus, ovary, melanoma, nasopharynx, larynx, thyroid, malignant brain, leukemia, Hodgkin’s lymphoma, non-Hodgkin’s lymphoma, and multiple myeloma. Stage reporting was less meaningful for the hematologic malignancies and brain; so initial reporting was for all patients with those histologies. For the other 19 histologies, data was presented for all, local, regional, and distant stages of disease, plus in situ was included for bladder cancer. More detailed analyses within each tumor type could be done for any of the other variables, such as small cell and non-small cell lung cancer, or different types of leukemia, as shown later in some examples. Both internal (Hoag vs Hoag) and external (Hoag vs SEER) comparisons included proportion in each stage and survival within each stage. This facilitated estimations as to how much earlier diagnosis and stage migration was influencing outcomes as opposed to changes in therapy. Examples of how data was reported is shown in (Table 3-6) and (Figure 1-3). This data was not previously unpublished.

Table 4:  Initial treatment for patients with invasive cancer (including in situ bladder) diagnosed during the successive eras 1989-1995 and 1996-2003.

Treatment	1989-1995 % of 7,662 patients	1996-2003 % of 12,977 patients	P value Chi Sq
Surgery only	33.5	30.5	<.0001
Surgery + radiation	8.2	7.1	.0034
Radiation only	7.2	6.8	.27
Systemic therapy only*	17.8	16.6	.026
Surgery + systemic therapy	11.8	10.6	.0081
Radiation + systemic	0.6	2.1	<.0001
Surgery, radiation & systemic	9.6	13.7	<.0001
Received any systemic therapy	39.8	43.0	<.0001
No anti-cancer therapy	11.3	12.5	.011

* chemotherapy, hormonal therapy, biological response modifier (BRM)

Table 3 illustrates comparisons to an internal benchmark based on the population of Hoag cancer patients diagnosed during 1995-2003 to those diagnosed during 1989 to 1995 and shows proportions of patients exhibiting various characteristics in the two successive eras. It can be readily seen that changes include increased Asian and Hispanic patients, increased elderly patients, and an increased proportion of patients with local disease at diagnosis and a corresponding decrease in proportion with distant disease at diagnosis, which was consistent with the increased emphasis on screening for early detection. Table 4 shows how treatments changed over time for patients diagnosed during 1995-2003 to those diagnosed during 1989 to 1995. From the data it was evident that fewer patients underwent surgery as the only initial treatment modality, and more patients received systemic therapy. This likely reflects changes in medical practice including increased emphasis on adjuvant therapy, and the introduction of new chemotherapy and biological therapeutic agents.

Table 5 shows the median and 5-year overall survival rates for patients diagnosed during 1995-2003 to those diagnosed during 1989 to 1995. A standard log rank test was used to test for significance of differences in survival curves. Based on the combination of patient numbers and p-values, the data suggested increases in survival overall and for patients diagnosed with lymphoma and cancers of the breast, prostate, and lung. Table 6 illustrates how comparisons of relative survival rates overall and by stage can be used to compare to a national benchmark, and how inferences from such a table can be hypothesized to explain the differences. In the example shown, which is for lung cancer, the relative survival was much better locally than nationally, overall and within each general stage of disease, and that a much higher proportion of patients had local disease at diagnosis. It is critical to include survival for all patients and not just by stage to exclude an erroneous interpretation of improved survival just because of stage migration, also known as the Will Rogers phenomenon [17].

Table 5: Intramural historical comparisons of observed 5-year survival rates by cancer types for consecutive time periods.

CANCER	1989-1995	1996-2003	1989-1995 Med Surv Mos	1996-2003 Med Surv Mos	1989-1995 5-yr surv %	1996-2003 5-yr surv %	P val KM
BREAST	1375	2391	NR	NR	82	86	.002
GYNECOLOGIC
Cervix	186	157	NR	NR	72	70	.753
Uterus	212	411	NR	NR	73	77	.147
Ovary(no borderline)	213	279	37	45	40	45	.180
GASTROINTESTINAL
Colon	479	797	64	80	51	56	.142
Rectal-RS	175	350	93	105	63	66	.472
Esophagus	71	97	5	11	14	17	.413
Liver & other biliary	44	120	5	7	5	15	.0008
Pancreas	176	263	6	8	3	6	.664
Stomach	84	149	13	16	14	19	.289
UROLOGIC
Prostate	1114	1915	NR	NR	80	88	<.0001
Bladder w/ in situ	292	487	NR	109	74	68	.683
Kidney & Renal Pelvis	140	251	72	90	51	55	.408
Testis	83	89	NR	NR	95	97	.545
HEAD & NECK
Larynx	61	68	NR	105	66	62	.593
Oral, Pharynx	147	226	76	99	55	60	.393
Thyroid	85	309	NR	NR	95	95	.928
LUNG	1082	1627	12	14	18	24	.004
MELANOMA of skin	313	595	NR	NR	78	80	.449
BRAIN	82	255	15	15	28	23	.642
HEMATOLOGIC
Leukemia	164	329	15	27	30	41	.015
Lymphoma-NHL	319	480	50	107	46	59	.0003
Myeloma	65	116	30	45	28	36	.116
Hodgkin’s	63	80	NR	NR	85	85	.928
ALL Invasive	7,662	12,977	87	120	56	62	<.0001

NR=not reached.

Figure 1 shows the relative survival of all patients in each of four successive eras. Similar curves were generated for observed survival and relative survival for each tumor type, and for those malignancies in which there were sufficient numbers of patients. Data also were generated by general stage within each specific malignancy. Such data clearly provided evidence for continuous improvement based on the survival metric. Figures 2&3 are examples of “report card comparisons” to internal and external benchmarks for quality assurance purposes. Figure 2 shows the change in percentage of Hoag patients surviving 5 years during successive time periods. Figure 3 shows changes in relative survival for Hoag patients compared to SEER data for patients diagnosed during the same time period. When data existed for a sufficient number of patients, more in-depth analyses were undertaken to better understand why these differences existed.

IV Quality Assurance and Quality Improvement: Overviews and Dashboards

Our initial interest in registry data was to address the question of whether survival had improved in the years after opening the cancer center to patient care (February 1991) compared to an earlier era. At that time there were many primary care physicians and surgeons who were pessimistic and cynical regarding proclaimed advances in cancer care, and who doubted that the existence of a comprehensive cancer center could improve outcomes and/or accelerate improvement in outcomes. We asked the question: did observed and relative survival increase after opening of the Hoag cancer center compared to internal and external benchmarks? As previously reported, because the registry had data from 1986 forward, and because of the specific SEER data available at that time, we compared outcomes for more than 5,000 patients diagnosed during 1986-1991 to the more than 10,000 diagnosed during 1992-1999 [18]. The data showed that there had been improvements in survival compared to internal benchmarks as well as compared to external benchmarks. For all patients, observed median survival improved by more than two years, and 5-year survival increased by four percentage points. There were significant increases in observed survival for cancers of the lung, breast, prostate, stomach, pancreas, and oral cavity. Relative survival for the Hoag patients was higher than SEER in both eras, and two percentage points higher in the more recent era. 5-year relative survival rates were higher in Hoag patients for only 50% of the 24 different tumor types during 1986-1991 compared to 88% for 1992-1997 [18].

We identified changes related to diagnosis and therapy that likely contributed to the improved survival, including an increase in the proportion of patients with local stage disease at diagnosis due to earlier detection, decreased regional disease at diagnosis because of more sensitive radiology and pathology detection, a decrease in unknown stage that may have contributed to better management decisions, an increase in patients diagnosed and/or treated at Hoag, increased use of multimodality therapy rather than surgery alone and the increased of systemic therapies (chemotherapy, hormonal therapy, bio-immunotherapy) of any type, and specifically bio-immunotherapies such as interferon, interleukin-2 approved in 1992 for renal cell and 1998 for melanoma, and monoclonal antibodies rituximab approved in 1997 for lymphoma, and trastuzumab for breast cancer in 1998. Of note, these improvements occurred even though a higher percentage of patients received no treatment because of increasing age of the cancer population and associated co-morbidities. Simple bar graphs were used to show the relative differences in survival for the intramural and extramural comparisons. As shown in (Figures 2 & 3), this became the format for a basic “dashboard” for each general tumor type, that was used in subsequent annual quality reports that involved comparisons to internal and external benchmarks.

V Quality Assurance Studies

Detailed studies were published for patients diagnosed with cancers of the breast, lung, colon, and prostate [19-23]. These cancer types were chosen because of the volume of such cases in the Hoag registry, and the numbers of deaths associated with these malignancies. A general report was generated limited to patients with each specific diagnosis, but detailed analyses were also performed based on subsets of patients defined by stage at diagnosis, and in the case of lung cancer, by specific histologic subtypes.

i Breast

The breast cancer study showed no difference in stage distribution between 1986-1991 and 1992-1997, but there was improved overall survival and improved survival for patients with local and regional stage disease [19]. This did not necessarily mean that the improvements were all related to therapy since within the local disease category tumors got smaller, and the use of sentinel lymph node testing resulted in detection of minimal regional disease; so, stage migration likely played a role in the stage-specific increased survival. In contrast, the extramural comparisons showed an increase in local stage disease and decrease in distant stage disease, and better relative survival within all stages compared to SEER. The associated therapy information for the internal comparisons showed a dramatic decline in the use of surgery alone with concomitant increases in the use of surgery plus adjuvant systemic therapy, and tri-modality therapy that included surgery, radiation, and systemic therapy. Another interesting graph showing relative survival for all breast cancer patients, showed that for every interval SEER had analyzed since 1981-1987, Hoag relative survival was actually lower during earlier comparisons, but then was the same during 1986-1993 (both 86%), but thereafter was progressively higher for Hoag patients such that for 1995-2001, Hoag relative survival was 98% compared to 88% for SEER.

ii Lung Cancer

Two large lung cancer analyses were conducted and published [20, 21]. After a pulmonologist questioned whether doing surgery on lung cancer patients over the age of 80 was warranted, a study was designed to examine the outcomes of such patients relative to other age groups. The sample was limited to patients who had undergone their surgical procedure at Hoag Hospital. A higher proportion of octagenarians had local stage lung cancer at diagnosis than younger patients, and a higher frequency of adenocarcinoma histology, but smaller proportions underwent surgical resection for local or regional disease [20]. There was no statistical difference among the survival curves for the different age groups, and the 5-year overall survival of 62% for the octagenarians was as good or better than other age cohorts except for those less than age 50 [20]. The 5-year relative survival for the octagenarians was over 90%. This study refuted the suggestion that older patients were undergoing unnecessary lung cancer surgery and that their life expectancies were being decreased because of such surgery. This study was reassuring to thoracic surgeons and impacted management decisions by pulmonologists and primary care physicians who often counseled lung cancer patients regarding the risk-to-benefit ratio of aggressive lung cancer treatment.

A subsequent lung cancer study was conducted to examine changes in survival over time. This study showed that median and 5-year survival rates for patients diagnosed with invasive lung cancer had steadily improved during each successive era [21]. One apparent reason for this improvement was the increased proportion of patients with local disease at diagnosis, which steadily increased in each era, in large part because of earlier diagnosis [21]. The median age of patients kept increasing with increasing proportions of adenocarcinoma and decreases in squamous cell and small cell histologist. In eras subsequent to 1986-1991, females constituted the majority of patients. In terms of treatment modalities, surgery alone was used increasingly because of more local stage disease due to earlier diagnosis and/or more indolent biologic cancer behavior in older patients, and there was greater use of surgery plus systemic therapy, and tri-modality therapy, and declining use of external beam radiation as a sole or adjunctive modality. Survival also increased relative to SEER data in each of four successive eras [21].

iii Colon Cancer

After national reports suggested that surgical removal and pathology identification of at least 12 lymph nodes was a quality indicator because of association with better survival, a study was performed to address the question: were 12 or more lymph nodes being detected in local colon cancer resections. The sample was limited to patients who had undergone their surgical procedure at Hoag Hospital. This study required data for individual patients with colon cancer (not including rectal) that was sorted for specific informational fields, the specific numbers of lymph nodes identified by pathologists in the pathology report, and the operating surgeon of record. The analysis covered 574 patients diagnosed during 1998 to 2005. One finding was that the average number of nodes identified increased from an average of 8.0 in years prior to 1998 to 14.5 in the subsequent era [22]. This improvement was attributed to specific changes in procedures and policies of the pathology department, which had been implemented with this goal in mind. This study established that the average number of nodes identified was more than 14, and that there were variations based on anatomic location, stage of disease and the experience of the operating surgeon [22].

iv Prostate Cancer

In tumor boards there was consistent disagreement regarding the value of post-operative radiation therapy in patients with localized prostate cancer. Therefore, a study was conducted that addressed the question of whether the addition of post-operative radiation therapy was associated with increased survival in patients with resection margin-positive prostate cancer. Over a 17-year period there were 96 margin-positive patients identified who did not receive radiation therapy and 17 for whom adjuvant radiation therapy was administered. The difference in 15-year survival was 100% vs 57% in favor of adding radiation [23].

VI Data to Supplement Other Clinical Studies

i Glioblastoma

The registry was used to define a contemporary control group of patients with glioblastoma for comparison to patients treated with an intralesional immune cell therapy [24].

ii Renal Cell Cancer

A published study addressed survival for renal cell cancer patients treated with high-dose interleukin-2 (IL-2) at the local hospital [25]. This report was timely because of the introduction of the targeted therapies for renal cell cancer.

iii Melanoma

One study addressed the question of long-term survival for melanoma patients who were treated with high-dose interleukin-2 (IL-2) at the local hospital and showed that these outcomes compared favorably to other published reports [26]. Another published study showed that melanoma patients treated with high-dose IL-2 had a better survival if they also were treated with investigational patient-specific autologous vaccines [27].

iv Breast

A clinical case report documented a complete response of breast ductal carcinoma in situ (DCIS) to neoadjuvant chemotherapy, but also included analyses related to the heterogeneity and chemosensitivity of DCIS in a larger cohort of patients derived from the registry data base, and the significance of high-grade in situ “comedocarcinoma” [28].

VII Other Benefits

Registrars expressed their personal gratitude that the data they were entering was being used to address specific questions related to the quality of care in the hospital in which they worked. They related that such awareness was associated with increased job satisfaction and increased pride in their work unit. Although not studied objectively, the impression was that registrars gave greater attention to accuracy of data entry when they knew that someone locally was working with the data they had entered rather than just looking for errors in interpretation of clinical information. An unpublished study of outcomes based on race/ethnicity overall for high-volume histologies, was helpful in obtaining a 6-year certification approval for the hospital’s continuing medical education.

Conclusion

It is the opinion of the authors that local use of cancer registry data is underutilized as a resource. Cancer registries house vast amounts of data that can be used to address issues of continuous quality improvement and quality assurance based on comparisons to internal and external data. If patient numbers are sufficient, analyses designed to address specific clinical questions can be sufficient for peer-reviewed publication.

Declaration

Supported by the Hoag Hospital Foundation and Hoag Hospital.

Abbreviations:

AACR: American Association of Central Cancer Registries
NAACR: North American Association of Cancer Registries
NCRA: National Cancer Registrars Association

Article Info

Article Type

Research Article

Publication history

Received: Mon 30, Dec 2019
Accepted: Wed 22, Jan 2020
Published: Mon 27, Jan 2020

Copyright

© 2023 Robert O. Dillman. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Hosting by Science Repository.
DOI: 10.31487/j.COR.2020.01.04

Author Info

Robert O. Dillman Stephanie E. McClure

Corresponding Author

Robert O. Dillman
Hoag Cancer Institute, Newport Beach, California

Figures & Tables

Table 1:  Miscellaneous Report Labels:  Categories and Content.

Title	Content
Site Code	Tumor type & number of cases
Total Cases	Total number of cases
Histology	Each pathology diagnosis and numbers of each and proportion of all diagnoses.  There can be numerous subtypes; so for subset analyses, best to group these together, which is facilitated by sorting alphabetically.  For instance, a lung cancer file had 48 histologies, but these could be grouped into adeno, squamous, neuroendocrine, small cell, and others. Also allows quality check to exclude histologies such as lymphoma and sarcoma that occurred in that site.
General Stage	Numbers and percentages by local, regional, distant, and unknown stage and substages of regional to include direct extentsion, lymph node involvement, or both.  This is assigned by registrars and is the staging used by SEER.
Summary AJCC Stage Group	Stages I, II, III, IV with A, B, C subsets.  Because of detail required, much higher percentage likely to be classified as unknown stage.  For instance in a lung cancer report, only 2% had an unknown general stage, but 20% had an unknown AJCC stage.  Furthermore, AJCC stage definitions have changed over time, making it hard to use for historical  comparisons.
Age at Diagnosis	Numbers and percentages by age. Easy to detect some errors.  For instance, in one lung report, 6 of 1,334 were listed as heaving a negative age. By sorting, can readily see age range.
Race	White, Black, Chinese, Japanese, Vietnamese, Korean, other Asian, Filipino, American Indian, other, and unknown.  Have to combine with ethnicity if want to do subset analysis on Hispanics and/or non-Hispanic Whites.
Accession Year	Shows number and percentage of cases by year of diagnosis and accession into the registry.
Grade	Undifferentiated/anaplastic, poorly-, moderately- , or well-, differentiated, or unknown.  Useful prognostic marker.
Class of Case	Non-analytical had already been excluded; so these were; [1] 1st DX here, Rx here, [2] 1st Dx elsewhere, Rx here, and [3] 1st DX here, Rx elsewhere. Useful to determine outmigration of patients for therapy, and referral of patients for therapy.
Dx Confirmation Code	Numbers and percentages by positive histology, positive cytology without histologic confirmation, radiographic diagnosis only, clinical diagnosis only, unknown, and others
Ethnicity	Non-Spanish, Spanish, Mexican , Cuban, etc. Can be combined with Race to create large groups such as Asian, Black, Hispanic and non-Hispanic White
Sex	Number and percentage by gender: male, female, and other
Primary Payer	HMO, PPO, Medicare, Medicaid, Uninsured, Medicare through managed care, Medicare with supplement, and others
Primary Medical Oncologist	Name and percentage of cases, including none. However, reflects MD of record at time of initial diagnosis, and not uncommon for patients to change physicians in the course of care, and sometime non-med oncs listed.
Primary Surgeon	Name and percentage of cases, including none.  Because of surgical reports at diagnosis, names listed are accurate
Rx Type	Treatments in order by number and percentage.  Needs to be sorted alphabetically to group.  Options include surgery, radiation, chemotherapy, hormonal therapy immunotherapy, and various combinations, and “other.”  Common inaccuracies in the systemic therapies listed, especially if not started until after completion of surgery and radiation and/or if treatment was given in physician’s infusion center rather than in a hospital-based facility.

Table 2: Content of survival data in Excel^® worksheets.

Categories	Content
Title  (top of page)	Description of data set or subset and years of inclusion
Interval month (Column A)	Consecutive number by months from date of diagnosis
Alive in month (Column B)	Number of patients known to be alive at the start of this month
Dead in month (Column C)	Number of patients known to have died during that month
Last seen in month (Column D)	Number of patients known to still be alive as of that month, but no subsequent follow up information
Exposed to Risk (Column E)	Computer calculation based on previous 3 columns for calculation used in actuarial survival figure
Proportion Dying (Column F)	Column C divided by Column B
Proportion Surviving (Column G)	Column B minus Column D, divided by Column B
Proportion at end of interval (Column H)	Calculation of the Kaplan-Meir statistic for overall survival (all-cause mortality)
Proportion relative survival (Column I)	Adjusted calculation of Column H to produce the relative survival statistic (adjusted for competing causes of death based on age and gender of those dying in the interval compared to expected rates for year in use.

Table 3: Characteristics of patients diagnosed with "invasive cancer* during the successive eras 1989-1995 and 1996-2003.

Category	1989-1995 n=7,662 patients	1996-2003 n=12,977 patients	Chi Square P value
Median Age in years	65.0	65.0	---
% “White” non Hispanic	94.4	92.2	<.0001
% classified as “Asian”	2.6	3.6	<.0001
% Hispanic	2.6	3.9	<.0001
% “African-American”	0.43	0.34	.30
% Female	52.0	52.9	.21
% < age 20 years	0.43	0.49	.53
% > age 90 years	0.97	1.39	.0080
% in situ bladder	2.0	2.2	.34
% local	46.8	50.2	<.0001
% regional	21.6	20.7	.12
% distant	23.6	20.6	<.0001
% unknown general stage	6.0	6.3	.39
% Diagnosed & treated at Hoag	83.2	83.5	.58
% Diagnosed elsewhere,  treated at Hoag	15.4	15.3	.84
% Diagnosed at Hoag, treated elsewhere	1.4	1.2	.23
% Diagnosed by histopathology	91.8	92.1	.45
% Diagnosed by cytology only	5.8	5.5	.38

*includes tumors classified as “unknown stage;” excludes “in situ” except for bladder.

Table 4:  Initial treatment for patients with invasive cancer (including in situ bladder) diagnosed during the successive eras 1989-1995 and 1996-2003.

Treatment	1989-1995 % of 7,662 patients	1996-2003 % of 12,977 patients	P value Chi Sq
Surgery only	33.5	30.5	<.0001
Surgery + radiation	8.2	7.1	.0034
Radiation only	7.2	6.8	.27
Systemic therapy only*	17.8	16.6	.026
Surgery + systemic therapy	11.8	10.6	.0081
Radiation + systemic	0.6	2.1	<.0001
Surgery, radiation & systemic	9.6	13.7	<.0001
Received any systemic therapy	39.8	43.0	<.0001
No anti-cancer therapy	11.3	12.5	.011

* chemotherapy, hormonal therapy, biological response modifier (BRM)

Table 5: Intramural historical comparisons of observed 5-year survival rates by cancer types for consecutive time periods.

CANCER	1989-1995	1996-2003	1989-1995 Med Surv Mos	1996-2003 Med Surv Mos	1989-1995 5-yr surv %	1996-2003 5-yr surv %	P val KM
BREAST	1375	2391	NR	NR	82	86	.002
GYNECOLOGIC
Cervix	186	157	NR	NR	72	70	.753
Uterus	212	411	NR	NR	73	77	.147
Ovary(no borderline)	213	279	37	45	40	45	.180
GASTROINTESTINAL
Colon	479	797	64	80	51	56	.142
Rectal-RS	175	350	93	105	63	66	.472
Esophagus	71	97	5	11	14	17	.413
Liver & other biliary	44	120	5	7	5	15	.0008
Pancreas	176	263	6	8	3	6	.664
Stomach	84	149	13	16	14	19	.289
UROLOGIC
Prostate	1114	1915	NR	NR	80	88	<.0001
Bladder w/ in situ	292	487	NR	109	74	68	.683
Kidney & Renal Pelvis	140	251	72	90	51	55	.408
Testis	83	89	NR	NR	95	97	.545
HEAD & NECK
Larynx	61	68	NR	105	66	62	.593
Oral, Pharynx	147	226	76	99	55	60	.393
Thyroid	85	309	NR	NR	95	95	.928
LUNG	1082	1627	12	14	18	24	.004
MELANOMA of skin	313	595	NR	NR	78	80	.449
BRAIN	82	255	15	15	28	23	.642
HEMATOLOGIC
Leukemia	164	329	15	27	30	41	.015
Lymphoma-NHL	319	480	50	107	46	59	.0003
Myeloma	65	116	30	45	28	36	.116
Hodgkin’s	63	80	NR	NR	85	85	.928
ALL Invasive	7,662	12,977	87	120	56	62	<.0001

NR=not reached.

Figure 1: Actuarial survival curves for patients diagnosed during 1998-2003 and 2004-2009.

Figure 2: Differences in percentage points for 5-year observed survival rates for patients diagnosed during 1998-2003 compared to 2004-2009 (internal benchmark comparison).

Figure 3: Differences in percentage points for 5-year relative survival rates between local and national populations (external benchmark comparisons) of cancer patients diagnosed during 2005-2011.

References

1. Brien SE, Dixon E, Ghali WA (2009) Measuring and reporting on quality in health care: a framework and road map for improving care. J Surg Oncol 99: 462- 466. [Crossref]
2. Miller TE, Gutmann VL (2009) Changing expectations for board oversight of healthcare quality: the emerging paradigm. J Health Life Sci Law 2: 33-77. [Crossref]
3. Russell TR (2006) Nothing to hide: transparency in healthcare. Med Gen Med 8: 83.
4. Porter ME (2009) A strategy for health care reform--toward a value-based system. N Engl J Med 361: 109-112. [Crossref]
5. Harlan LC, Hankey BF (2003) The surveillance, epidemiology, and end-results program database as a resource for conducting descriptive epidemiologic and clinical studies. J Clin Oncol 21: 2232-2233. [Crossref]
6. Gloeckler Ries LA, Reichman ME, Lewis DR, Hankey BF, Edwards BK (2003) Cancer survival and incidence from the Surveillance, Epidemiology, and End Results (SEER) program. Oncologist 8: 541-552. [Crossref]
7. Satcher MJ, Litton AG, Waterbor JW, Brooks CM (2009) The Journal of Cancer Education: A retrospective review of quality indicators. J Cancer Educ 4: 16-21. [Crossref]
8. Desch CE, McNiff KK, Schneider EC, Schrag D, McClure J et al. (2008) American Society of Clinical Oncology/National Comprehensive Cancer Network Quality Measures. J Clin Oncol 26: 3631-3637. [Crossref]
9. Hutchison CL MH, Burch M, Gottschalk R (2004) Cancer Registry Management: Principles and Practice, 2nd edition. Dubuque, Iowa: Kendall/Hart Publishing Co.
10. USCODE- title 42 (1998) The Public Health and Welfare: chap6A-subchaII-part M: National Program of Cancer Registries-sec 280.
11. Edge SB, Compton CC (2010) Cancer Staging Handbook from the AJCC Cancer Staging Manual. 7th edition. eds. Springer 17: 1471-1474.
12. Fleming ID (2001) AJCC/TNM cancer staging, present and future. J Surg Oncol 77: 233-236. [Crossref]
13. EDERER F, AXTELL LM, CUTLER SJ (1961) The relative survival rate: a statistical methodology. Natl Cancer Inst Monogr 6: 101-121. [Crossref]
14. Henson DE, Ries LA (1995) The relative survival rate. Cancer 76: 1687-1688.
15. Siegel RL, Miller KD, Jemal A (2019) Cancer statistics,2019. CA Cancer J Clin 69: 7-34. [Crossref]
16. Brenner H, Hakulinen T (2009) Implications of incomplete registration of deaths on long-term survival estimates from population-based cancer regist ries. Int J Cancer 125: 432-437. [Crossref]
17. Feinstein AR, Sosin DM, Wells CK (1985) The Will Rogers phenomenon. Stage migration and new diagnostic techniques as a source of misleading statistics for survival in cancer. N Engl J Med 312: 1604-1608. [Crossref]
18. Dillman RO, Chico SD (2005) Cancer patient survival improvement is correlated with the opening of a community cancer center: comparisons with intramural and extramural benchmarks. J Oncol Pract 1: 84-92. [Crossref]
19. Dillman RO, McClure SE (2007) Improving survival for patients with breast cancer compared with intramural and extramural benchmarks. Clin Breast Cancer 7: 480-485. [Crossref]
20. Dillman RO, Zusman DR, McClure SE (2009) Surgical resection and long-term survival for octogenarians who undergo surgery for non-small-cell lung cancer. Clin Lung Cancer 10: 130-134. [Crossref]
21. Dillman RO, McClure SE (2014) Steadily improving survival in lung cancer. Clin Lung Cancer 15: 331-337. [Crossref]
22. Dillman RO, Aaron K, Heinemann FS, McClure SE (2009) Identification of 12 or more lymph nodes in resected colon cancer specimens as an indicator of quality performance. cancer 115: 1840-1848.  [Crossref]
23. Dillman RO, Hafer R, Cox C, McClure SE (2011) Overall survival benefit from postoperative radiation therapy for organ-confined, margin-positive prostate cancer.  Int J Radiat Oncol Biol Phys 79: 719-723. [Crossref]
24. Dillman RO, Duma CM, Ellis RA, Cornforth AN, Schiltz PM et al. (2009) Intralesional lymphokine-activated killer cells as adjuvant therapy for primary glioblastoma.  J Immunother 32: 914-919. [Crossref]
25. Dillman RO, Barth NM, VanderMolen LA, Fong WH, Mahdavi KK et al. (2011) Should high-dose interleukin-2 still be the preferred treatment for patients with metastatic renal cell cancer? Cancer Biother Radiopharm 26: 273-277. [Crossref]
26. Dillman RO, Barth NM, VanderMolen LA, Mahdavi K, McClure SE (2012) Should high-dose interleukin-2 still be the preferred treatment for patients with metastatic melanoma? Cancer Biother Radiopharm 27: 337-343. [Crossref]
27. Dillman RO, Depriest C, McClure SE (2014) High-dose IL-2 in metastatic melanoma: better survival in patients immunized with antigens from autologous tumor cell lines. Cancer Biother Radiopharm 29: 53-57.[Crossref]
28. Dillman RO, VanderMolen LA, Heinemann FS, McClure SE (2015) Complete regression of ductal carcinoma in situ with chemotherapy and trastuzumab: should we reconsider management of high-grade comedo ductal carcinoma in situ?  MOJ Clin Med Case Rep 2: 00035.