2020 technical documentation for licensure and workforce ... · survey response rates and bias...

www.FlCenterForNursing.org

2020 Technical Documentation for Licensure and Workforce Survey

Analysis

2

CONTENTS Data Extract ............................................................................................................................................................. 4

Data Cleaning .......................................................................................................................................................... 5

Licensure Data Cleaning ................................................................................................................................ 5

APRN + RN Licenses ................................................................................................................................... 5

Deactivated Nurses ..................................................................................................................................... 7

Address Data Cleaning ................................................................................................................................... 7

Nurse Placement in Counties and Regions ........................................................................................ 7

Workforce Survey Data Cleaning ............................................................................................................... 8

Identifying the Potential Nurse Workforce ................................................................................................. 9

Survey Response Rates and Bias Analysis................................................................................................ 10

Estimation of the Actual Nurse Workforce .............................................................................................. 12

Conclusions and Limitations of the Data Sources ................................................................................. 14

References ............................................................................................................................................................ 16

Appendix A: 2018-2019 Workforce Survey Questions ....................................................................... 17

Appendix B: County Composition of FCN Regions ................................................................................ 21

June 2020 Technical Documentation for Licensure and Workforce Survey Data 3

Technical Documentation for Licensure and Workforce Survey Analysis

BACKGROUND The Florida Center for Nursing (Center) was established in 2001 to address issues related to Florida’s nurse supply, demand, and shortages (Florida Statute 464.019(10)). Since 2006, the Center has obtained nurse licensure data from the Florida Board of Nursing (FBON) for analyses directed in statute. The FBON licensure database contains the most complete information specific to the regulation of Florida nurses (e.g. license status/eligibility to practice, demographic characteristics, and last known address). FBON licensure data do not include information about the work behavior of nurses, e.g. whether nurses are working (within or outside nursing), how much they work, or in what setting. Since 2008, the Center has partnered with FBON and Medical Quality Assurance (MQA) to integrate a voluntary Workforce Survey into the online nursing licensure renewal process. The Workforce Survey generates important data for workforce analysis, such as work status, hours worked, and highest degree held. Measurements include those established by the National Forum of Nursing Workforce Centers minimum data set. Using a unique file number as an identifier, workforce survey data are merged with licensure data. Because the Center is primarily interested in the amount of nursing labor provided in Florida, licensure and survey data are cleaned and subset to isolate nurses who could reasonably be practicing nursing within the state of Florida. We call this subset the potential nurse workforce. To be included in the potential nurse workforce, licensees must hold a clear and active (or military/temporary military active) license indicating eligibility to practice with no disciplinary restrictions. Licensees must also have a last known address indicating they are practicing and/or living in Florida (if not practicing out of state). A substantial majority of renewing nurses participate in the Workforce Survey each year, but many cases still lack workforce data. In addition to those who do not complete the voluntary survey during renewal, nurses newly licensed in Florida are not exposed to the Workforce Survey until they renew their licenses for the first time.1 The Center uses available licensure information for non-respondents (e.g. practicing address and demographic characteristics) to estimate employment status and comparisons to survey participants, when possible. This document provides technical details about the process of merging, cleaning, and estimating values for some cases using weighting and imputation techniques based on licensure and Workforce Survey data.

1 In the 2018-2019 cohort, these groups comprised approximately 32.5% of the potential nurse workforce. Renewing nurses who did not participate in the survey comprise 18.7% of the potential workforce, and newly licensed, clear and active nurses represent 13.8% of the potential workforce. However, due to individual circumstances, some nurses newly licensed between 2018 and 2019 may have renewed during the same biennial cycle. Among the newly licensed nurses in the potential workforce (n = 51,440), 79% did not provide a survey response. This proportion fluctuates each cycle. In 2016-2017, none of the newly licensed nurses completed a survey.


DATA EXTRACT Following the completion of a two-year renewal cycle, the Center extracts data from static files received from MQA. This data extract includes records for each nursing license held in Florida by Registered Nurses (RN), Advanced Practice Registered Nurse (APRN), and Licensed Practical Nurses (LPN). Renewal cycles are based on the licensee’s expiration date of either April 30th or July 31st.2 In even years (e.g. 2018), approximately two-thirds of RNs and APRNs are expected to renew between January and July. In odd years (e.g. 2019), the remaining one-third of RNs and APRNs renew from January through April and all LPNs renew from March through July. Data used for current analyses intend to represent all renewals and new licenses between January 1, 2018 and December 31, 2019. The Center receives data snapshots on the 21st of each month. For analyses of the 2018-2019 cohort, the Center compared and combined December 2019 and January 2020 extracts to best represent the population of licensees as of December 31st.3 This cutoff represents the transition to the next biennial renewal cohort, beginning on January 1 of the next even year (e.g. 2020). Licensure, address, and survey data are merged using an MQA generated file number as the unique identifier to join records. Though surveys are completed throughout the two-year period, we treat survey data as representing the best possible estimate of a nurse’s work status at the end of the renewal cycle to provide the best possible estimate of the workforce as of January 2020. This approach is not without limitation yet remains the most accessible method of analyses given current resources available. Each record in the potential nurse workforce is assigned an estimated work status based on their survey responses or reported practicing addresses when responses are not provided. Imputation techniques are used to further estimate working status among the total population. Practicing addresses are provided through the MQA renewal process. A full-time equivalent (FTE) value is estimated for all survey respondents and weighted to more closely represent the total population, including non-respondents.

2 With the exception of Temporary Military Active licenses which expire exactly one year from the date received. Nurses practicing in the state of Florida with non-expired temporary military licenses are included in this sample. 3 The December 21st data extract is the primary source of data. To avoid excluding any renewal records and survey responses between December 21st and December 31st, data from the January extract are incorporated when a) no record is found for that nurse in the December file or b) the nurse’s license or active status changed between December and January. In both situations, only nurses whose records indicate a license expiration date consistent with the current cohort of interest are included. For instance, 2018-2019 renewals should have an expiration date of 2020 or 2021. Nurses whose expiration date was after July 31, 2021 as of the January data extract were excluded from the sample and will be analyzed with the next cohort. This approach provides the added benefit of capturing renewals and new licenses through the end of the calendar year, allowing for inclusion of new licensees graduating in December, and allows nurses who missed their renewal deadlines an additional five to seven months to complete renewal for inclusion in the sample. This approach is new as of the 2018 reporting cycle and may affect response rates.


DATA CLEANING In partnership with MQA, the Center has made improvements to the survey instrument to increase data validity and reduce cleaning (e.g. skip/display logic, rejection of implausible values, etc.). However, due to the biennial nature of each cohort, implementation of changes requires time. More revisions are needed during the 2020-2021 cycle. Due to changes in licensure and workforce survey databases, data cleaning efforts vary and unique challenges for each cohort must be faced to maintain consistency for comparisons. Licensure Data Cleaning

Licensure data are cleaned for implausible dates of birth and dates of initial licensure. Implausible values are typically a result of MQA values representing missing data (e.g. January 1, 1901). Missing data generated by cleaning procedures are typically minimal. In the 2018-2019 cohort, only 18 of the 476,232 records in the December/January licensure data extract have missing dates of original licensure, and 44 records were recoded to missing due to implausible dates of birth. Naturally occurring missing data also exist for other variables in the licensure database.4 About 1% of all records were missing data on gender. Beginning in the 2018 survey cycle, the workforce survey includes race/ethnicity questions. Survey values supplemented responses for about half of the 3% of licensees with missing race data, reducing the proportion of all records with missing race data to about 1.5%. Additional variables are created to identify nurses who maintain a Florida license and those that meet criteria to be included in the potential workforce based on criteria outlined by the MQA License Status Definitions.5 APRN + RN Licenses Effective October 1, 2018, the Advanced Registered Nurse Practitioner (ARNP) certification transitioned to an Advanced Practice Registered Nurse (APRN) license in Florida. Additionally, Clinical Nurse Specialist (CNS) licenses were transitioned to an APRN license (Florida Board of Nursing, 2018). Following this change, the Florida Board of Nursing automatically converted valid ARNP and CNS licenses to a new APRN license. The Florida Department of Health established a new APRN record for all ARNP and CNS licensees that met criteria for conversion and converting existing ARNP or CNS license records to an active RN license. This resulted in a “rank change” of sort, establishing RN records for an estimated 35,055 advanced practice nurses which needed to be filtered out so as to not overestimate the number of RNs in the potential workforce, or RNs working as nurses in Florida, as these are duplicate records for APRN nurses counted separately.

4 Prior to 2016-2017, the Center’s analyses included only cases with non-missing values for a given variable. The current analysis attempts to include cases with missing values, when possible, to most accurately represent the proportion of participants with a given characteristic. Inclusion of cases with missing values are notated in the report, when applicable. This may impact comparisons of previous years. 5 https://appsmqa.doh.state.fl.us/MQASearchServices/LicStatus.html

https://appsmqa.doh.state.fl.us/MQASearchServices/LicStatus.html


This conversion impacted the Center’s tracking of total number of licensees as well as the gains and losses to the potential RN and advanced practice workforce. New data cleaning processes were developed to more closely approximate the number of people eligible to work in Florida, rather than the number of licenses held. Among the 37,201 records provided to the Center in April 2020, filtered to best represent all Florida APRNs as of December 31, 2019…

1) 31,750 maintained an RN and an APRN license record and had the same license number for each record. These represent ARNP records that automatically transitioned to an APRN license AND had their ARNP license “downgraded” to an RN license on the technical transition date following the statutory effective date of October 1, 2018.

2) 4,403 maintained an RN and an APRN license record for which the license numbers for each record were distinct, resulting from RNs applying for an APRN license after the bulk conversion of ARNP/CNS to APRN licenses.

3) 1,048 maintained an APRN license but did not hold a distinct RN license, indicating nurses who applied for an APRN license in Florida after receiving their RN licensure in another state.

A total of 35,055 APRN records were matched to a December 2019 RN record and were removed from the population of RNs. The remaining 1,098 RN records comprising the sum of groups 1 and 2 above were either already removed from the database (e.g. null and void, deactivated) or had an original RN licensure date on or after January 1, 2020.6 Moving forward, nurses eligible for an APRN license may choose to retain or relinquish their RN licenses. Beginning in this survey cycle, upon licensure renewal, nurses with multiple Florida licenses would only be required to complete the Nurse Workforce Survey once. When a survey was completed and linked with licensure data not considered active based on the criteria above, values were recoded to associate the survey response with the active license. Reported gains and losses to the potential workforce as well as counts of total RNs and APRN licensees were impacted by this conversion process. APRN gains and losses were not able to be calculated for the first time due to the combined impact of…

1) Changes to the profession code for all APRN licenses 2) Changes to the file number and license number for many, but not all licenses 3) Duplication of RN records matching APRN records where one has a distinct license/file

number and the other does not, but no other identifiable differences. 4) Changes to license status/attainment which may have occurred between December 2019

(cohort snapshot date) and April 2020 when the Center received the FDOH comparison table. 5) Inconsistent availability of the FDOH unique identifier which is likely to be more accessible

for the wider population in the future.

6 What remains concerning about the latter group (n = 30) is why their APRN license would have an original licensure date in 2018 or 2019 while their date of original RN licensure is after this time, based on the criteria explained to the Center by FDOH. This group remains in the APRN database as they are minimal and meet all other criteria required.


All efforts to identify gains and losses between the 2017 and 2019 databases resulted in numerous errors which did not allow for confidence in estimates without the risk of duplication or mislabeling of cases. Deactivated Nurses Florida joined the Nurse Licensure Compact (NLC) in 2018, resulting in the establishment of a multistate license which enables licensed practical and registered nurses to be eligible to practice in any member state. Multistate licenses are issued by the nurse’s “home state” – the state in which they maintain primary residence. The Center is charged by Florida statute to assess Florida’s involvement in NLC. In April 2020, the Center learned that FDOH uses the license code “Deactivated” to identify nurses who relinquished their Florida license in favor of a multistate license issued by another home state. FDOH sent a current list to the Center in April 2020. The Center developed best estimates of those deactivated prior to January 2020 and added them to the supply database (n = 10,502) to be able to identify this as a valid “loss reason” distinguished from other “null and void” dropped cases. The removal of RN/APRN duplicate records and the addition of the Deactivated nurses to the supply database resulted in 451,679 total records in the database, including records that do and do not maintain active Florida licenses. Address Data Cleaning

Additionally, many records contain incomplete address data and data entry errors, such as zip code values that do not match their reported city and/or state. When possible, many of these errors are manually fixed in the data cleaning process to place addresses into the correct counties. Each licensee’s recorded zip code is compared against a SAS statistical software lookup table of zip codes which matches the data extract as closely as possible.7 In the cases where given zip codes are invalid, additional address information provided is used to look up the most likely county match. The process typically creates a county placement for several Florida addresses which were previously unplaced and may have otherwise been dropped from the sample as having no valid Florida address. Location data are essential to the Center’s counting efforts, yet this process is particularly time intensive and the assumptions required for cleaning impose a risk of inaccurate assignment. The Center suggests that MQA/FBON implement efforts to validate/auto-fill address data input by licensees to ensure accurate and up-to-date values. Nurse Placement in Counties and Regions The extensive effort to establish accurate addresses is also the foundation for regional reporting. 8 Stakeholders depend on local data for nurse workforce planning, so the Center

7 Zip codes change over time but are reasonably steady in the short term. SAS makes new zip code lookup tables available quarterly at http://support.sas.com/rnd/datavisualization/mapsonline/html/misc.html 8 In previous versions of the Workforce Survey, participants were asked to provide practicing and/or home counties and survey data would supersede address data from the licensure file. Efforts to minimize time spent on the survey resulted in the elimination of this question in 2014. Thus, regional and county information is based entirely on addresses provided to MQA/FBON. Since it is unknown how recently address fields in the

http://support.sas.com/rnd/datavisualization/mapsonline/html/misc.html


assigns nurses who were last known to be living and/or working in the state of Florida to counties, regions, and local workforce development boards (LWDB)9 to prepare regional reports on request. There are 67 counties and 24 LWDBs in Florida. See Appendix B for a map of Florida designating counties by region and LWDB. The licensure database contains two sets of address fields used to place nurses into Florida regions/counties: a mailing address (where nurses wish to receive mail from the FBON) and a practice address. Once address data are cleaned to identify the most accurate mailing and practicing counties, each nurse is assigned to a primary county and Florida address status. For primary county, region, and LWDB assignment, preference is first given to valid practicing addresses (including those out of state). When practicing addresses are unknown, or “not practicing” is indicated, primary regional assignments are based on mailing addresses (including those with out of state mailing addresses). Licensees with no valid mailing or practicing address information are coded as missing in their regional placement but remain in the sample if they hold an active Florida license. In the 2018-2019 cohort…10

• 102,958 were not living and/or working in FL and/or were not eligible to practice • 159 had a valid Florida license, but no known addresses • 30,754 had a Florida mailing address, but weren’t working or did not report a practice address • 310,005 had Florida mailing and practicing addresses • 7,803 had Florida practicing addresses and non-Florida mailing addresses

This effort maximizes the accuracy of placement for describing workforce distribution as much as can be expected despite data limitations. Address data do not guarantee certainty of where nurses live and work, although they do allow identification of nurses who report working or receiving mail outside of Florida. Since mailing addresses may be a home residence or a work location – wherever the nurse wishes to receive mail from the FBON – precision is difficult to obtain for nurses who do not have valid and distinct addresses in both fields. County placement may be less accurate for describing patterns of nurses commuting from one county to another to work (or working in several locations). Yet generalized understandings can be estimated at the regional level. Workforce Survey Data Cleaning

Each survey cycle, the Center actively works to reduce survey response errors (e.g. using skip/display logic to remove irrelevant questions and reduce incompatible responses). When necessary, workforce survey data are transposed and cleaned to adjust implausible

licensure database have been updated, the accuracy of regional placement should be interpreted with caution. Regional questions were reimplemented into the workforce survey in 2020. Preliminary results regarding the effectiveness of this change will be available in the 2022 report. 9 A classification used by CareerSource Florida for workforce planning and outreach efforts. https://careersourceflorida.com/wp-content/uploads/2018/03/Local-Workforce-Development-Boards.png 10 Represents the total population (n = 451,679). Counts may vary from those presented elsewhere as some discrepancies exist between licensure, address, and survey values regarding employment. Self-reported employment status supersedes addresses when available.

https://careersourceflorida.com/wp-content/uploads/2018/03/Local-Workforce-Development-Boards.png


values (e.g. when hours worked per week exceeds the number of hours in a week or when the number of weeks worked per year exceed the number of weeks in a year). Efforts are also made to resolve incompatible values, particularly among those that impact estimated employment status (e.g. working status is missing but hours, weeks, setting, and position are provided) or FTE status (e.g. participants reports they work full-time but did not provide hours worked per week or weeks worked per year.) While these errors/cleaning may become less common as survey revisions are implemented, discretion is used to modify values when a plausible assumption can be made. Others are recoded to or left as missing. Challenge in workforce survey data that emerged during this renewal cycle included…

1) APRN survey responses linked to a matching RN license. These were recoded to link to APRN license as duplicate RN licenses were removed from the database (see following section).

2) FDOH database conversion impact survey response formats and the Center had to merge files from two distinct systems/formats. When duplicate surveys were identified between the legacy system (2018) and the current system (2019), the more current survey overrode the legacy version to assume more updated employment information.

IDENTIFYING THE POTENTIAL NURSE WORKFORCE A multi-step process is used to generate a subset from the total file representing the potential nurse workforce: those eligible to work as nurses and providing a Florida address. First, nurse records are excluded if their license and active status indicate that they do not maintain a Florida license (e.g. retired, delinquent, suspended). 11 Next, nurses who appear to be living or working out of state are excluded. In addition to having a Florida mailing and/or practicing address, licenses must be clear and active to be included in the potential nurse workforce. Records indicating any other status (e.g. inactive, conditional, obligations) are removed from the sample.12 Figure 1 details the number of licensees excluded in each step of this process for the 2018-2019 cohort. Categories for removal may not be mutually exclusive (e.g. someone without a Florida address may also have an inactive license), therefore counts of nurses removed at each stage may not encompass the true counts of every nurse with a given status.

Figure 1: 2018-2019 Licensees Excluded from Analysis, by Reason

11 Based on the Florida Department of Health, Medical Quality Assurance definitions of license statuses 12 In 2018-2019 cohort, includes 566 nurses that are considered “eligible to practice” with restrictions, according to MQA definitions of license statuses (e.g. Obligations, Probation, Emergency Restrictions)


In the 2018-2019 cohort, the average age of excluded nurses was 52.0, compared to 47.2 in the potential workforce. While this is a prevalent gap, excluded nurses also include those who remain in the licensure database with a “Retired” license status, which will likely skew their age upward compared to those eligible to practice. Approximately 68% of excluded nurses were white, and about 87% were female. Clear and active nurses in the potential workforce were slightly less likely to be white (60%). About 12% of nurses in the potential workforce were male, compared to 11% of those excluded. These comparisons do not differentiate between newly licensed and renewing nurses within each group (excluded vs. potential workforce), as some differences between group may be impacted by differences within group. In some years, newly licensed nurses are completely excluded from analysis of survey data as newly licensed nurses have a significantly lower chance of being exposed to the workforce survey. In the 2016-2017 renewal cycle, there were no survey participants who were newly licensed during that cycle. In contrast, about 31% of the nurses in the potential workforce with an original license date in 2018-2019 submitted a survey response. Each cycle, discretion is required to determine whether analyses will be exclusive to renewing nurses or more representative of the total potential workforce. Figure 2 highlights the total 2018-2019 potential nurse workforce by new/renew status and survey participation. Figure 2: 2018-2019 Potential Workforce by Renewal Status and Survey Participation

SURVEY RESPONSE RATES AND BIAS ANALYSIS The online Workforce Survey was available for completion between January 1, 2018 and December 31, 2019. Nurses who renewed their licenses during this time were given the option to complete this survey during the online renewal process.13 During this cycle, changes were made within the survey database to implement skip/display logic, new questions, and other MQA database management changes outside the control of the Center. During each cycle, unique data management challenges must be assessed although the Center attempts standardization to the maximum extent to allow comparisons over time. Per Figure 2 above, a substantial majority of nurses (87%) participated in the survey.14 Renewing nurses had a 97% response rate, an estimated increase of about 16% since

13 In 2008, the Florida Board of Nursing began mailing postcard reminders for license renewal, instead of paper renewal forms. The vast majority of nurses have been renewing their licenses online since this time, as paper renewal forms must now be specially requested. 14 Comparisons of response rates prior to 2016-2017 may show decreases in recent years. However, changes are more likely attributed to newer data cleaning procedures which keep more nurses in total eligible population, rather than an indication of a decreased participation rate.


the previous cycle. About 31% of newly licensed nurses participated in the survey. Table 1 describes 2018-2019 survey participation by renewal group and rank. Among nurses newly licensed between 2018 and 2019, there was substantially higher response rates among those whose license expires in 2021 (Table 1). When paired with substantially higher response rates among LPNs, this is likely explained by LPNs initially licensed in 2018, as they would be required to renew in 2019. These circumstances make them simultaneously a newly licensed and a renewing licensee within the same cycle and explains the increased likelihood that they would be exposed to the Nurse Workforce Survey. APRN response rates are also particularly high for newly licensed nurses, which may be attributed to adjustments to original licensure dates resulting from the ARNP to APRN conversion or exposure to the survey on an RN license that was dropped as a duplicate. Table 1: 2018-2019 Workforce Survey Response Rates, by Renewal Group and Rank

Renewal Group Potential Workforce Total New Renewing Group 1 (exp. April 2020) 8.6% 98.3% 88.5% Group 2 (exp. July 2020) 15.3% 98.7% 85.7% Group 3 (exp. April 2021) 41.5% 97.5% 88.3% Group 4 (exp. July 2021) 54.8% 92.4% 87.6%

License Type LPN 54.8% 92.4% 87.6% RN 15.4% 98.2% 86.6% APRN 84.6% 73.6% 94.2%

Total 31.4% 97.1% 87.5% Responses from newly licensed nurses remain in the dataset when possible, although limited knowledge can be obtained specific to this group of nurses. High response rates among renewing nurses increase the generalizability of the responses to the overall population, particularly for LPNs and RNs. To examine potential patterns of bias, we compared characteristics of survey respondents to those that did not participate in the survey (Table 2, next page). Nurses who participated in the survey are more likely to be white, compared to non-participants, and a slightly higher proportion of non-participants are male. Survey participants were about nine years older, on average, than those that did not take the survey. Participants have also held a Florida nursing license more than three times as long as non-participants, on average. Differences in age and length of licensure likely result from fewer newly licensed nurses being exposed to the Nurse Workforce Survey. Differences between average age and years licensed in Florida were statistically significant between groups (p < 0.001), although significance may be attributed (in part) to large sample sizes. These differences indicate that our survey data may underrepresent younger and more diverse nurses and should highlight the importance of increasing response rates among these groups over time.


Table 2: 2018-2019 Demographic Comparison of Respondents and Non-Participants

Non-Participants

Survey Respondents

Race/Ethnicity Asian 5.3% 5.6% Black 17.9% 17.2% Hispanic 17.9% 13.4% White 49.9% 61.3% Other 2.4% 2.4%

Gender Female 83.2% 87.7% Male 14.1% 11.8%

Age Group 30 or younger 36.8% 10.0% 31-40 24.6% 22.4% 41-50 16.0% 22.8% 51-60 11.2% 23.0% 61-70 8.3% 17.9% 71 or older 3.2% 3.9%

Average Age 39.3 48.4 Years Licensed in FL 3.9 13.4 Note: Proportions may not equal 100% due to missing data

ESTIMATION OF THE ACTUAL NURSE WORKFORCE Despite high participation rates, non-response does occur due to the voluntary nature of this survey. In order to estimate the total number of working nurses in Florida, regardless of survey participation, working status and full-time/part-time status were estimated when not provided. Simple proportion weights were developed based on available (or estimated) characteristics for the entire potential nurse workforce (n = 346,672).15 Weights were applied to relevant survey questions to improve representativeness to the total potential workforce.16

Weight = Population Proportion Sample Proportion

The first step was to determine employment status of all members of the potential workforce sample (working or not working). For survey participants, this was primarily determined by

15 Because licensure data for the entire potential workforce is known, and since all nurses have an equal probability of participating in the survey, weights were only created to account for non-response, rather than sampling bias (although survey accessibility and mobile friendliness should be explored in the future). 16 Weighting was incorporated into licensure and workforce survey for the first time in 2016-2017. Some other states that have used weighting in healthcare workforce survey analyses include Virginia (n.d.), Utah (Harris & Ruttinger, 2017), and California (Spetz, Chu, Jura, & Miller, 2017).


survey responses indicating employment, and supplemented by the state/zip code of their practicing address or hours reported when employment status was missing. For nurses who did not respond to the survey, employment status was first estimated by reported practicing address. When working status could not be approximated, hot deck imputation was used to assign the remaining nurses with missing data to a category of working or not working.17 In the 2018-2019 cohort, 83.2% of the potential workforce, regardless of survey participation, indicated or appeared to be employed at the time of renewal, including 83.2% of renewing nurses and 83.6% of newly licensed nurses. Approximately 84% survey participants in the potential workforce were working, compared to an estimated 78% of non-responders in the potential workforce. When applicable, weighted frequencies were applied to survey data to improve the likelihood that the sample is representative of the population of renewing nurses, by adjusting for non-response. Weights were calculated for each working status by age group and rank. Table 3 provides an example of the weights, for RN nurses in 2018-2019:

Table 3: 2018-2019 RN Weights by Employment Status and Age Group

Employed by Age Population Proportion (%)

Sample Proportion (%) Weight

Working 30 or younger 56.6% 41.2% 1.37 31-40 years old 86.4% 84.7% 1.02 41-50 years old 82.2% 85.3% 0.96 51-60 years old 79.6% 86.1% 0.93 61-70 years old 52.2% 57.2% 0.91 71 or older 7.4% 8.0% 0.93

Not working a 0.0% 30 or younger 6.2% 3.0% 2.05 31-40 years old 10.7% 9.2% 1.16 41-50 years old 10.1% 9.9% 1.02 51-60 years old 13.8% 14.4% 0.96 61-70 years old 23.3% 24.8% 0.94 71 or older 10.5% 10.6% 1.00

a Not working includes proportions of licensees when working status remains missing following cleaning/imputation efforts (n = 1,015)

Estimated weights were also used to identify the proportion of nurses with a given full-time equivalency (FTE) employment status.18 Survey respondents provided information on the

17 Analyses of cohorts prior to 2016-2017 involved random selection and placement of non-responders into dichotomous employment grouping based on calculated proportions of working survey participants using a matrix of age by gender by license type (Florida Center for Nursing, 2010). 18 Using weights to estimate FTE for all renewing nurses was introduced in 2018 analyses of 2016-2017 renewal cohort. This approach is a vastly different from prior analyses which involved a series of mean


number of hours they worked per week and weeks worked per year. This information was used to assign an FTE value with the following formula:

FTE = (hours/week × weeks/year) 1,872

In this formula, the numerator represents the hours worked per year by the respondent, and the denominator represents the minimum hours worked in a year if a nurse represents 1.0 FTE. The typical number of hours per year used in computations like this is 2,080, which is based on 40 hours worked per week for 52 weeks. However, nurses who work 36 hours a week (i.e. three 12-hour shifts) are considered full time employees, therefore the minimum hours per year for 1.0 FTE status represents 36 hours per week for 52 weeks (1,872 hours). Nurses working more than 1,872 hours per year were capped at 1.0 FTE, while those working fewer than 1,872 hours per year were assigned an FTE fractional value. When a nurse reported he or she was not working in nursing, their FTE was assigned a value of 0. Some additional data cleaning was required. For instance, some participants described their work status as full time, but indicated they worked 8 hours per week. When possible, records were reviewed on an individual basis to identify if any clearer estimates could be determined based on other responses. Although the FTE value for members of the potential nurse workforce is unknown if they did not complete a workforce survey, the use of imputation and weighted proportions among survey respondents increased the ability for the sample data to represent the estimated proportions of working and FTE status among the total population. In the 2018-2019 cohort, this resulted in an estimated 176,517 working nurses with 1.0 FTE, and an overall average weighted FTE of .91 for all working nurses. CONCLUSIONS AND LIMITATIONS OF THE DATA SOURCES All analyses of the Florida nurse supply based on licensure and Workforce Survey data inevitably suffer from some degree of missing or inaccurate data. The Center’s process for cleaning the data, assigning nurses into counties and regions, and imputing missing data attempts to correct some of the data problems which, if left unchecked, would distort our view of the nurse supply. The exclusion process used to identify the potential nurse workforce generates the best estimate of nurses who could be working in Florida, including their location in a specific region of Florida. However, it is important to reiterate that licensure data do not indicate whether nurses are working in the field of nursing or how much they work. If clean Workforce Survey data are available for a nurse, it is a straightforward process to determine whether and where a nurse practices nursing. However, due to the voluntary and self-report nature of this survey, population parameters are estimated using imputation and weighting techniques. While these efforts improve the

substitutions and a reliance on categorical survey data recoded to the midpoints to represent hours worked. This shift in methodology may affect comparisons to previous years. For a full explanation of the methodology used in the past, see the Florida Center for Nursing’s (2010) Technical Document.


survey data by providing statistical adjustments for non-response, the information for individual cases may not be accurate. The only way to provide completely accurate information regarding the total nurse supply in each cohort is to implement additional mandatory fields in the license renewal process while continuously improving database technologies to ensure the most reliable data entry. Without this, assumptions about the population must be gleaned from estimates among self-selected survey participants, data point cleaning, and locations based on last known address in the licensure database. The incorporation of the Workforce Survey beginning in 2008 has improved data quality and facilitated efforts to accurately quantify the nursing workforce. While there will always be at least some missing data, possessing complete workforce information on a substantial majority of renewing nurses is a huge benefit for nurse workforce analysis and planning in Florida. The Center is appreciative of our continuing collaboration with the Florida Board of Nursing and Medical Quality Assurance during each renewal cycle and will continue to do our part to improve the quality of the Workforce survey, while making efforts to retain the ability to provide trend analysis and maintain the National Forum of State Nursing Workforce Centers’ National Nursing Workforce Minimum Dataset (2016) for nurse supply.


REFERENCES Florida Board of Nursing. (2018, October 4). Advanced Practice Licensure Change. Retrieved

from Florida Board of Nursing: https://floridasnursing.gov/advanced-practice-licensure-change/

Florida Center for Nursing. (2010). Technical Documentation for Licensure and Workforce Survey Data Analysis. Retrieved from https://www.flcenterfornursing.org/DesktopModules/Bring2mind/DMX/Download.aspx?Command=Core_Download&EntryId=384&PortalId=0&TabId=151

Florida Department of Health, Division of Medical Quality Assurance. (2018). License Status Definitions. Retrieved from https://appsmqa.doh.state.fl.us/MQASearchServices/LicStatus.html

Harris, A., & Ruttinger, C. (2017). Utah's Advance Practice Registered Nurse Workforce, 2017: A Study on the Supply and Distribution of APRNs in Utah. The Utah Medical Education Council. Retrieved from https://nursing.utahmec.org/wp-content/uploads/APRN_2017_FINAL.pdf

Healthcare Workforce Data Center (HWDC). (n.d.). HWDC Methodology. Virginia Department of Health Professions. Retrieved from https://www.dhp.virginia.gov/hwdc/docs/MethodologyandGlossary.pdf

Spetz, J., Chu, L., Jura, M., & Miller, J. (2017). California Board of Registered Nursing 2016 Survey of Registered Nurses. Retrieved from http://www.rn.ca.gov/pdfs/forms/survey2016.pdf

The National Forum of State Nursing Workforce Centers. (2016). Minimum Nurse Supply Dataset. Retrieved from http://nursingworkforcecenters.org/wp-content/uploads/2016/11/National-Forum-Supply-Minimum-Dataset_September-2016.pdf

April 2020 Technical Documentation for Licensure and Workforce Survey Data 17

APPENDIX A: 2018-2019 WORKFORCE SURVEY QUESTIONS Note: Display logic added mid-cycle in January 2019. Additional efforts include replacing manual data entry fields with drop down box selections and minor wording changes to increase participants’ accurate interpretation of questions. Nurse Licensure Questions (Added 2018) Q01. Do you have a Florida only nursing license or a multi-state license?

Multi-state Florida Only

(Display 1a and 1b if 1 = Multistate) Q01a: In which state do you work the most hours per week?

(Drop down box with 50 states plus DC) Q01b: In which other states have you worked in the last two years? (Please select

all that apply) (Drop down box with 50 states plus DC)

Q02. Year of Initial U.S. Licensure Select Year Q03. In what country were you initially licensed as an RN or LPN? Select Country Q04. What type of nursing degree/credential qualified you for your first U.S. nursing license?

Vocational/Practical Certificate – Nursing Diploma – Nursing Associate Degree – Nursing

Baccalaureate Degree – Nursing Master’s Degree – Nursing Doctoral Degree – Nursing

Q05: What is your highest level of education in NURSING?

Vocational/Practical Nursing Certificate Diploma in Nursing Associate Degree in Nursing Baccalaureate Degree in Nursing

Master’s Degree in Nursing PhD in Nursing Doctorate of Nursing Practice Other Nursing Doctoral Degree

Q06. (Added in 2019) Have you completed any non-nursing degrees (after high school)?

Yes No (Display if Q06=Yes) Q06a: What is your highest NON-NURSING degree?

Associate Degree – Non-Nursing Baccalaureate Degree – Non- Nursing Master’s Degree – Business Related Master’s Degree – Health Related Master’s Degree – Other Law Degree (JD)

Doctorate in Medicine (MD, DO) Doctoral Degree – Other Health Discipline Doctoral Degree – Other Discipline No Degree Outside Of Nursing


Q07. Are you credentialed to practice as one of the following Advanced Practice Nurse certifications?

Certified Registered Nurse Anesthetist Certified Nurse Midwife Nurse Practitioner (Any Specialties) None of the above

Nursing Employment Questions Q08. Are you actively employed for pay in one or more nursing position or in a position that requires a nursing license?

Yes No

Display questions Q09-15 if Q08 = Yes Q09. In how many positions are you currently employed as a nurse?

1 2 3 or more

Q10. Which of the following best describes your main nursing position? Your main position is the one at which you work the most hours during your regular work year.

Full-time Part-time Per diem

Q11. How many hours do you work during a typical week in all your nursing positions? (For per diem or part time employees who do not work regularly, please provide the approximate number of hours per week during the weeks that you do work).

(Drop down box, range 0 - 80)

Q12. In total, how many weeks do you work during a typical year in all your nursing positions? (Includes paid time off) (Note: year round employment = 52 weeks).

Drop down box, first option: Year round (52 weeks) Followed by numeric range, 1-51 weeks)

Q13. Please identify the type of setting that most closely corresponds to your main nursing practice position.

Hospital Nursing Home/Extended Care Assisted Living Facility Home Health Correctional Facility Academic Setting Public Health Community Health Healthcare Consulting/ Product Sales Urgent Care/Walk-in Clinic Dialysis Center

School Health Service Occupational Health Hospice Ambulatory Care Setting Insurance Claims/Benefits Policy/Planning/Regulatory/Licensing Agency Physician’s Office Temporary / Staffing Agency Other Telehealth


Q14: Please identify the position title that most closely corresponds to your main nursing practice position. Staff Nurse Advance Practice Nurse Nurse Executive/Administrator Nurse Manager Nurse Faculty Quality Management/Risk Management Utilization Review/Infection Control Nurse Researcher (Non-Faculty)

Consultant Travel Nurse Case Manager Educator Other-Health Related Other-Not Health Related

Q15: Please identify the employment specialty that most closely corresponds to your

main nursing practice position. Critical Care Adult Health Family Health Anesthesia Cardiology Community Emergency-Trauma Geriatric/Gerontology Home Health Information Technology Maternal-Child Health Medical Surgical Neonatal Nephrology Occupational Health

Oncology Operating Room Palliative Care/Hospice Pediatrics Peri-Operative Psychiatric/Mental Health/Substance Abuse Rehabilitation (non-psychiatric) School Health Telehealth Women’s Health/OB-GYN Other Acute Care Other

Display Q16 if Q09 = No Q16. If not currently employed for pay in nursing, please select the option that best describes your status? (Select only one)

Seeking Work as a Nurse Seeking Work in a Field Other Than

Nursing Not Seeking Work at This Time

Retired Employed in a non-nursing position

Display if Q16 = Not Seeking Work at this Time Q16a. If not currently seeking employment for pay, please indicate the reasons. (Select all that apply.)

Taking care of home and family Currently enrolled in school Disabled/Illness

Difficulty in finding a nursing position Inadequate Salary Other

Non-Nursing Employment

Q17. Do you work any hours for pay in a field other than nursing? Yes No


(Display if Q17=Yes) Q17a. Which of the following best describes your non-nursing position?

Full-time Part-time Per diem Other Nursing Employment Q18. Do you perform any nursing work as a volunteer?

Yes No

Q19. What are your nursing employment plans for the next 5 years? (Select all that apply.)

Work in nursing as much as I do now Work in nursing less than I do now Work in nursing more than I do now Advance my nursing education/

nursing credentials

Move into Florida Move out of Florida Leave nursing/retire Other/Don’t know

Demographics Q20. Race (select all that apply)

American Indian or Alaska Native Asian or Asian American Native Hawaiian or Pacific Islander White Black or African American Other (please specify) ________

Q21. Do you identify as Hispanic or Latino/a?

Yes No Q21a. (DISPLAY IF Q21 = Yes) Please Specify (select all that apply)

Mexican or Mexican American Puerto Rican Cuban or Cuban American Other Hispanic, Latino/a, or Spanish Origin


APPENDIX B: COUNTY COMPOSITION OF FCN REGIONS

2020 technical documentation for licensure and workforce ... · survey response rates and bias...

Documents