Supplementary Material for “Wait, Do I Know This Person?” Understanding Misdirected Email

Emilee Rader and Anjali Munasinghe. “Wait, Do I Know This Person?”: Understanding Misdirected Email. CHI 2019, Glasgow, UK, May 2019. DOI: 10.1145/3290605.3300520

1 Survey Overview

1.1 Data Collection

1.1.1 When did the survey take place?

Data collection for the survey took place starting on 2018-03-20 and ending on 2018-04-02.

1.1.2 How many responses were collected?

The survey was started by 943 potential respondents. After data cleaning, there were 380 respondents in the dataset.

1.1.3 How long did it take to complete the survey?

The survey took an average of 12.94 minutes for respondents to complete, including the consent and screening questions. The maximum completion time was 80.75 minutes, and the minimum was 3.25 minutes. Respondents had 24 hours from the time they began the consent form to complete the survey, and were told this in the instructions.

Completion Time Descriptives (in minutes)
Min	3.2
Median	10.5
Max	80.8
M	12.9
SD	9.9

1.2 Sample

1.2.1 How many respondents were ineligible vs. excluded?

Respondents were recruited by Qualtrics using their panel service, with quotas for gender (50% men and 50% women) and age (18-29: 25%, 30-49: 38%, 50-64: 21%, 65+: 15%). The age quota was based on information from the Pew Research Center’s Internet/Broadband Fact Sheet from Feb. 5, 2018 about the age distribution of US adults who use the internet, and data from the US Census Bureau’s 2016 American Community Survey about the age distribution of the United States population.

The Pew results tell us what proportion of different age groups use the internet (e.g., 66% of US adults who are 65+ are internet users). But that doesn’t tell us how many people in the US are 65+ years old, so we can’t calculate what proportion of our respondents should be 65+ just using the Pew data. From the Census ACS data, we know that in 2016, the most recent year for which data had been published at the time this research was conducted, 15% of the US population is 65+. So now we know that 66% of the 15% of the US population who are 65+ use the internet. The ACS also includes the information that there are about 323,127,515 people in the US, so using information from both sources we can calculate reasonable target percentages for survey recruiting.

Eligible respondents also were older than 18 and younger than 120 years old, and indicated that they had at least one email account on a popular free email service. The text of the question was, “Do you currently regularly use at least one email account from a popular free email service provider like Gmail, Outlook Mail or Hotmail, Yahoo! Mail, AOL Mail, etc.?”

Respondents who did not consent and who did not meet the age, gender, and email account screening criteria were excluded after declining consent or after providing an ineligible answer to one of the screening questions.

1.2.2 For what reasons were eligible respondents excluded?

• duplicate name and IP address: During data cleaning, we discovered that two respondents had taken the survey multiple times, based on the IP address associated with their responses, the first and last names they provided, and their answers to demographic questions being duplicated across multiple completed surveys. We kept the first completed survey from each respondent in the dataset, and removed the duplicates.

• failed attention check or finished too quickly: The survey included an attention check question and a check for finishing the survey too quickly. The check for “speeding” excluded responses from respondents who finished the survey in 1/3 the median time to complete, based on the “soft launch” of the survey. This works out to 3.54 minutes. The attention check consisted of asking respondents for their username (only the part before the @ sign) from their email address on a popular free email service, and then asking again for the same username later in the survey. If these did not match, they were directed to the end of the survey without being allowed to finish it. Respondents were told in the consent form and instructions that there would be “questions and other methods designed to ensure you are paying attention”. And, the first time they entered their username, they were asked to enter it and then re-enter it, and weren’t allowed to proceed until the usernames matched, to ensure that people weren’t kicked out of the survey due to a typo in the username.

• poor quality name or username: The survey included a question asking particpants to provide their first and last name and email username only (the part before the @ sign) for the free email service they belonged to. The survey included the instruction that this information would be used only for using similarity metrics to compare their name to their username, and would then be deleted. (“This information will be used to determine how similar your username is to your name, and will be deleted as soon as the comparison is complete.”) The name and username responses were inspected by hand to identify any that did not follow the instructions, and these responses were excluded. Some examples of this are responses that were only initials, gibberish (e.g. “ggvv”, “Th”, “dslkdasklklsd”), and responses that were not names (e.g. “boobs”, “none”, “I don’t know”, “refused”).

• poor quality short answer: The survey included two short answer questions, one asking them to explain why they would pick the new username that the entered in a previous question (variale name: “why_username”“), and the other asking them to describe an example of a wrong email that they had received (variable name:”wrongemail_example“). These responses were examined by hand and those that did not answer the question were excluded. We retained (did not exclude) cases with responses that contained an answer to the question, but then added spaces or punctuation or other text to meet the length requirement of the text entry field. Here are a few examples of excluded responses to the”why_username“” question:

  • “I wont be creating a new account, unless i want to. Man, bump these sentence. Ill make this short if i want ro.”
  • “Idk what to say and ……………………………..hsjsjsjsjsjjsnxjsjsjsjjddjndjsjsjsjdjxnxjdjsjxjjdjdn. Jenxjsnskxisnxjdj”
    
  • “Nothing at all don’t know nothing at all don’t know nothing at all don’t know nothing at all don’t know nothing at all don’t”
    

• “good or full” familiarity with fake word: The internet literacy question consists of a set of internet-related terms. Respondents were asked to specify their level of familiarity with the terms, ranging from “None” to “Full” familiarity. One of the words in the set was a fake word that respondents should not be familiar with. Respondents who said they had “Good” or “Full” familiarity with the fake word were excluded after completing the survey.

• incomplete survey: Respondents who started but didn’t finish the survey at the time the survey was closed and the dataset was downloaded from Qualtrics.

Here is the number of responses excluded for each reason:

excluded	n
did not consent or ineligible	229
failed attention check or finished too quickly	208
attention2 fake word good or full	59
poor quality name or username	30
poor quality short answer	28
incomplete survey	4
duplicate name and IP address	3
over quota	2

1.3 Respondent Demographics

The text of the survey questions about respondent demographics are included in the Demographics Block section of this document.

1.3.1 Age, Gender, Ethnicity

The average age of respondents was 44.99 (SD=16.81). Age ranged from 18 to 85. There were 196 women 181 men who participated in the study.

Age Descriptives (in years)
Min	18
Median	43
Max	85
M	45
SD	17

A large majority of respondents were white. Note that the ethnicity question allowed respondents to “choose all that apply”; in the first “ethnicity” table below “TRUE” indicates that the respondent checked the box, and NA indicates that they did not.

ethnicity	counts
white	308
hispanic	21
black	47
asian	14
native_am	7
middle_eastern	0
pacific	2
other	2

1.3.2 Region, Income, Education

Here are the characteristics of the sample in terms of region of the country, income, and education level:

1.3.3 Internet Literacy

The internet literacy variable consists of questions that are based on the Web Use Skills survey reported in Hargittai and Hsieh (2011). Respondents were asked the following question about the internet-related terms: “How familiar are you with the following Internet-related terms? Please rate your understanding of each term below from None (no understanding) to Full (full understanding)”.

We created a composite variable by averaging the self-reported responses for each respondent across the eight internet literacy items. The overall mean was 2.59 (SD = 0.89, median = 2.62). Cronbach’s alpha (a measure of internal consistency): 0.83.

The responses to each item are shown in the graphs below:

2 Survey Questions

Note: This section presents the complete survey instrument, along with descriptive statistics for each question. Formatting and pagination of the questions in the actual survey as it was administered differs from how the questions are presented in this document. Text shown in italics was not part of the survey.

2.1 Screening Block

The three questions below were asked after respondents had consented, but before starting the actual survey, to determine eligibility to participate. Respondents were ineligible if they reported being younger than 18 or older than 120 years old, and if they said “No” or “I’m not sure” to the email account question. The age and gender questions were used to meet the survey recruiting quotas.

• What is your age in years? [fill in the blank]

• What is your gender? [Man, Woman, Other (fill in the blank), Prefer not to disclose]

• Do you currently regularly use at least one email account from a popular free email service provider like Gmail, Outlook Mail or Hotmail, Yahoo! Mail, AOL Mail, etc.? [No, Yes, I’m not sure]

2.2 Email Accounts Block

Instructions: The survey should take approximately 15 minutes to complete. You have up to 24 hours to finish the survey, starting from the time you started reading the consent form on the previous page. Please note that the study includes questions and other methods designed to ensure that you are paying attention. If you do not pay careful attention to every question while you are completing the survey, you will be directed to the end of the survey without being able to complete it.

2.2.1 Reasons for creating a new email account

This question is based on Bentley et al.’s CHI 2017 paper “If a person is emailing you, it just doesn’t make sense”: Exploring Changing Consumer Behaviors in Email.

For which of the following reasons have you ever created a new email account? Please select all that apply:

• To have a more professional sounding username
• Old account getting too much spam
• Didn’t want to give “real” email address to a business
• Didn’t want to give “real” email address to a person
• To sign up for a service that requires a specific email provider such as Google Docs (gmail) or Xfinity (Comcast)
• Wanted to keep personal emails separate
• Problems with old account (locked out, deleted for non-use, etc.)
• To use for giving out to websites, such as online shopping sites
• To try something new
• Graduated from school and needed a new email address
• Started a new job and got a new email address from my employer
• Created a new account for a specific project, business or hobby
• Other (please fill in the blank)
• None of the above

reasons	count	percent
Wanted to keep personal emails separate	142	37.4
Old account getting too much spam	135	35.5
To have a more professional sounding username	94	24.7
To use for giving out to websites, such as online shopping sites	86	22.6
To sign up for a service that requires a specific email provider	82	21.6
Problems with old account	74	19.5
Didn't want to give 'real' email address to a business	68	17.9
To try something new	59	15.5
None of the above	46	12.1
Created a new account for a specific project, business or hobby	46	12.1
Didn't want to give 'real' email address to a person	42	11.1
Started a new job and got a new email address	40	10.5
Other reason	16	4.2
Graduated from school and needed a new email address	15	4.0

2.2.2 How many email accounts does each respondent have?

In the space provided below, please list the email accounts that you currently frequently use, starting with the email account you use the most often. If you use several email accounts about equally, please list up to five of them. Please do NOT enter the complete email address for each account. Instead, give each email account a nickname or a short phrase that describes it. For example, “personal email I’ve had the longest”, “work email”, “school email”, etc.

• Email account #1 (1) [fill in the blank]
• Email account #2 (2) [fill in the blank]
• Email account #3 (3) [fill in the blank]
• Email account #4 (4) [fill in the blank]
• Email account #5 (5) [fill in the blank]

Displayed only if the respondent entered something into all five blanks in the previous question:
Do you have more email accounts than the five you listed in the previous question? [No, Yes, I’m not sure]

More than Five Accounts	n
No	17
Yes	8
NA	355

2.3 Email Accounts Loop Block

Note: These questions were repeated once for each email account listed above. While all respondents filled in at least one blank, not all respondents filled in more than that. The NAs are removed from the graphs below.

2.3.1 How long have respondents had each account?

How long have you had the email account “${lm://Field/1}”?

• Less than one year
• 1-2 years
• 2-4 years
• 4-6 years
• 6-10 years
• More than 10 years
• I’m not sure

2.3.2 How did respondents choose their usernames?

When you created the email account “${lm://Field/1}”, how did you choose the username for that account? (The username is the part of the email address before the @ sign.) Select the response below that most closely matches your experience:

• I created the exact username I wanted.
• A username was assigned to me, and I kept it.
• A username was assigned to me, but I changed it later.
• The system suggested a username to me when I created the account, and I accepted it.
• Most of the usernames I tried to create were already taken, so I ended up with this one.
• I don’t remember.
• None of the above.

2.3.3 How early on did respondents create their email accounts?

At the time you created your “${lm://Field/1}” account, about how soon was it after the email service first became available? Please choose the response that best represents what you remember:

• Very soon after
• Somewhat soon after
• A while after
• Somewhat long after
• A long time after
• I don’t know

2.4 Names and Naming Block

2.4.1 Characteristics of respondents’ names

Instructions: The following question asks for three pieces of information, your first name, your last name, and ONLY the username (the part before the @ sign) of your email account from a popular free email service provider like Gmail, Outlook Mail or Hotmail, Yahoo! Mail, or AOL Mail that you use most often. This information will be used to determine how similar your username is to your name, and will be deleted as soon as the comparison is complete.

Note: descriptive statistics about respondents’ names were calculated, and edit distance comparisons between names and usernames were also computed. Then the raw name data were deleted from the dataset. Only the descriptive statistics were preserved, to protect respondents’ privacy.

Please enter the requested information below, and make sure to check carefully for typos:

• First Name [fill in the blank]
• Last Name [fill in the blank]
• Email Username [fill in the blank]

Please re-enter your First Name, Last Name, and the Email Username that you entered above:

• First Name [fill in the blank]
• Last Name [fill in the blank]
• Email Username [fill in the blank]

First name and last name length in number of characters:

Name Length (nchars)	Min	Median	Max	M	SD
firstname_lower_nchars	3	5	11	5.6	1.6
lastname_lower_nchars	2	6	14	6.4	1.8

2.4.2 Respondents’ self-report estimate of how common their name is

In your opinion, how common or uncommon is your FIRST name?

• Extremely common
• Moderately common
• Slightly common
• Neither common nor uncommon
• Slightly uncommon
• Moderately uncommon
• Extremely uncommon

In your opinion, how common or uncommon is your LAST name?

• Extremely common
• Moderately common
• Slightly common
• Neither common nor uncommon
• Slightly uncommon
• Moderately uncommon
• Extremely uncommon

2.4.3 Characteristics of respondents’ usernames, and suggested new username

Please imagine that you are creating a new email account on a brand new email service that currently has very few users, and you can have any email username that you want. What username would you choose? Please enter it below: [fill in the blank]

Username length (nchars), and whether usernames have digits or special characters:

variable	Min	Median	Max	M	SD
username_parsed_nchars	3	11.0	32	11.57	4.73
new_username_parsed_nchars	2	9.5	25	9.96	3.85
username_digits	0	1.0	1	0.52	0.50
new_username_digits	0	0.0	1	0.42	0.49
username_specials	0	0.0	1	0.11	0.31
new_username_specials	0	0.0	1	0.06	0.23

2.4.4 Why did people say they’d choose a particular username?

Note: the text in this section describes the content analysis of respondents’ reasons for choosing their usernames. The survey question is presented below. The remaining text in this section was not part of the survey.

Survey question: Please explain why you would choose the username “${choose username/ChoiceTextEntryValue}” if you were creating an account on a brand new email service that currently has very few users? Your answer must be at least 100 characters long, which is about 2-3 sentences. [fill in the blank]

We made a first pass through the responses and developed a set of categories of reasons why they would choose a particular username. These “reasons” reflect concerns they had, potential future uses of their email account, and characteristics they wanted their username to have. Then, two coders coded the responses using these categories.

Below are the codes, and the inter-rater reliability (Fleiss’ kappa):

Name and Birthdate

• Own Name: Their own name (kappa = 0.91)
• Own Birthdate: Their own birth date, year, etc. (kappa = 0.91)
• Own Nickname: Their own nickname (kappa = 0.91)

Life Characteristics

• Friend Name: The name of a friend or family member (kappa = 0.59)
• Location: A location, neighborhood, zip code, phone number (kappa = 0.87)
• Proper Noun: A proper noun that is the name of something (like a pet, fictional character, sports team or band, etc.) (kappa = 0.55)

Personal Meaning

• Meaningful: Some kind of personal meaning (like about their life, activities, interests or personality) (kappa = 0.53)
• Fits Me: I just like it, it just fits me, it just came to mind, etc. (kappa = 0.49)
• Catchy: It’s catchy, creative, fun, etc. (kappa = 0.72)
• Unique: It’s unique, uncommon, unusual, unlikely, etc. (kappa = 0.77)

Future Uses

• Memorable: Easy to recognize or remember (kappa = 0.81)
• Same ad Before: It’s the same username as they used in the past, or on other systems (kappa = 0.7)
• Professional: Professional or business related (kappa = 0.55)
• Security: Mentions privacy, identity theft, hacking, anonymity, spammers (kappa = 0.79)

None of the above (mutually exclusive) (kappa = 0.43)

The table and graph below show counts for each of the codes which had inter-rater reliability above 0.5, and on which either one of the coders said yes, that motivation was present.

code	n
Memorable	198
Own Name	114
Meaningful	112
Unique	65
Same as Before	56
Professional	39
Proper Noun	34
Security	27
Own Birthdate	27
Own Nickname	27
Catchy	19
Friend Name	14
Location	9

The categories were not mutually exclusive, except where indicated in the list of categories, above (fake response or none of the above). The categories were further grouped into four high-level sets of categories as shown in the bar chart. The tables below present counts of how many distinct respondents gave reasons that fell into each category.

Future Uses	n
TRUE	238
FALSE	142

Name and Birthdate	n
TRUE	144
FALSE	236

Life Characteristics	n
TRUE	56
FALSE	324

Personal Meaning	n
TRUE	171
FALSE	209

Name and Birthdate, Personal Meaning	n
TRUE	307
FALSE	73

2.4.5 What respondents consider when creating a new username

What is the most important thing for you to consider when creating a new email username? Please drag the following options to put them in order according to how important they would be for you:

• My new username should be easy for others to remember
• My new username should be similar to my real name
• My new username should be the same as my username on other accounts
• My new username should be unique compared to other people’s email usernames
• My new username should represent some aspect of my interests or personality
• My new username should sound professional
• My new username should be hard for spammers to guess

Question	Min	Median	Max	M	SD
My new username should be hard for spammers to guess	1	4	7	4.1	2.0
My new username should sound professional	1	5	7	4.5	2.0
My new username should be similar to my real name	1	4	7	3.9	2.0
My new username should be easy for others to remember	1	4	7	3.7	1.9
My new username should represent some aspect of my interests or personality	1	4	7	3.9	2.0
My new username should be the same as my username on other accounts	1	4	7	4.3	2.0
My new username should be unique compared to other people's email usernames	1	3	7	3.6	2.0

Bar charts of the ranks assigned to each item:

2.5 Misdirected Email Block

2.5.1 Questions about prevalence of misdirected email

The response items in the question below are our speculations based on piloting our interview protocol about possible symptoms of misdirected email, and reasons why misdirected email may happen. These were also screening questions for the interview study. Respondents were asked, “Which of the following statements describe you?” and instructed to select all that apply.

Which of the following statements describe you? Please check all that apply:

• I have received an email message that seemed like it was intended for someone else
• I have been asked in an email to confirm an account that I don’t remember creating
• I have sent an email message to the wrong person by mistake
• I have given out a “throwaway” email address that was fake or did not belong to me to a website, store, or restaurant, or in exchange for a service like free wifi
• I have used a “throwaway” email address that was fake or did not belong to me after being asked to sign up for an online account, mailing list, or other online service
• None of the above

Response	n	percent
None of the above	152	28
I have received an email message that seemed like it was intended for someone else	104	19
I have been asked in an email to confirm an account that I don’t remember creating	103	19
I have sent an email message to the wrong person by mistake	89	16
I have used a 'throwaway' email address that was fake or did not belong to me to sign up for an online account	50	9
I have given out a 'throwaway' email address that was fake or did not belong to me to a website...	47	9

2.5.1.1 I have received an email message that seemed like it was intended for someone else

Display if “I have received an email message that seemed like it was intended for someone else” was selected:
When was the last time you received an email message that seemed like it was intended for someone else?

• Today
• Within the past few days
• Within the past week
• Within the past month
• More than a month ago
• I’m not sure

How often do you receive email messages that seem like they are intended for someone else?

• Several times per day
• About once per day
• A few times per week
• About once per week
• About once per month
• Less than once per month
• I’m not sure

2.5.1.2 I have been asked in an email to confirm an account that I don’t remember creating

Display if “I have been asked in an email to confirm an account that I don’t remember creating” was selected:
When was the last time you remember being asked in an email to confirm an account that you didn’t remember creating?

• Today
• Within the past few days
• Within the past week
• Within the past month
• More than a month ago
• I’m not sure

How often are you asked in email to confirm an account that you don’t remember creating?

• Several times per day
• About once per day
• A few times per week
• About once per week
• About once per month
• Less than once per month
• I’m not sure

How many distinct respondents answered yes to the two misdirected email questions (“I have received…” and “I have been asked…”)?

Received Misdirected Email	n
TRUE	164
FALSE	216

2.5.1.3 I have given out a “throwaway” email address that was fake or did not belong to me to a website, store, or restaurant, or in exchange for a service like free wifi

Display if “I have given out a ‘throwaway’ email address that was fake or did not belong to me to a website, store, or restaurant, or in exchange for a service like free wifi” was selected:
When was the last time you gave out a “throwaway” email address that was fake or did not belong to you to a website, store, or restaurant, or in exchange for a service like free wifi?

• Today
• Within the past few days
• Within the past week
• Within the past month
• More than a month ago
• I’m not sure

How often have you given out a “throwaway” email address that was fake or did not belong to you to a website, store, or restaurant, or in exchange for a service like free wifi?

• Several times per day
• About once per day
• A few times per week
• About once per week
• About once per month
• Less than once per month
• I’m not sure

2.5.1.4 I have used a “throwaway” email address that was fake or did not belong to me after being asked to sign up for an online account, mailing list, or other online service

Display if “I have used a”throwaway" email address that was fake or did not belong to me after being asked to sign up for an online account, mailing list, or other online service" was selected:
When was the last time you remember using a “throwaway” email address that was fake or did not belong to you when you were asked to sign up for an online account, mailing list, or other online service?

• Today
• Within the past few days
• Within the past week
• Within the past month
• More than a month ago
• I’m not sure

How often have you used a “throwaway” email address that was fake or did not belong to you when signing up for an online account, mailing list, or other online service?

• Several times per day
• About once per day
• A few times per week
• About once per week
• About once per month
• Less than once per month
• I’m not sure

How many distinct respondents answered yes to the two “throwaway email address” questions (“I have used…” and “I have given out…”)?

Given out throwaway address	n
TRUE	69
FALSE	311

2.5.1.5 I have sent an email message to the wrong person by mistake

Display if “I have sent an email message to the wrong person by mistake” was selected:
When was the last time you sent an email message to the wrong person by mistake?

• Today
• Within the past few days
• Within the past week
• Within the past month
• More than a month ago
• I’m not sure

How often do you send email messages to the wrong person by mistake?

• Several times per day
• About once per day
• A few times per week
• About once per week
• About once per month
• Less than once per month
• I’m not sure

2.5.2 Have you ever received a “misdirected email”?

Display if “I have received an email message that seemed like it was intended for someone else” was selected:
We use the phrase “misdirected emails” to refer to email messages that are not spam, but are authentic email messages that just seem like they were meant for someone else. Have you ever received a “misdirected email”?

• Yes, I have received a “misdirected email”
• No, I have not received a “misdirected email”
• I’m not sure

Received Misdirected Email	n
Yes	54
No	28
I'm not sure	22
NA	276

2.5.3 Senders of misdirected email

Display if Yes, I have received a misdirected email was selected:
Who have you received “misdirected emails” from? Please select all that apply:

• Someone I know personally
• A person I don’t know who is from an organization I belong to, like work, school, or a volunteer group
• Someone who is a complete stranger
• A business, bank, online store, or other organization
• Other: (please fill in the blank)
• None of the above

Sender of misdirected email message is...	n
Someone who is a complete stranger	28
A business, bank, online store, or other organization	19
A person I don't know who is from an organization I belong to, like work, school, or a volunteer group	13
Someone I know personally	10
None of the above	1

2.5.4 Attention check question

Note: this question was used as an attention check question. If the usernames did not match, the respondent was excluded, and the following message was displayed: “You are seeing this message because you chose an incorrect answer to a question designed to ensure you are reading every question carefully. This means that you cannot complete the rest of the survey. If you have any questions or concerns, please contact Dr. Emilee Rader at emilee@msu.edu.”

In an earlier question, you were asked to enter your first name, last name, and ONLY the username (the part before the @ sign) of your email account from a popular free email service provider like Gmail, Outlook Mail or Hotmail, Yahoo! Mail, or AOL Mail. Please enter that same Email Username again below (be sure not to include any extra spaces or characters): [fill in the blank]

2.5.5 Types of misdirected email

Display if Yes, I have received a misdirected email was selected:
Below are different types of “misdirected emails” that you may have received. Please select all of the types of “misdirected emails” that you remember receiving in the past:

• Email from a children’s website asking for parental consent, for a child that you don’t know
• Email containing a statement or bill from an account that does not belong to you
• Email from someone you know in real life that was sent to you by mistake
• Account activation, registration or confirmation email for something you did not sign up for
• An email that contains a personal message meant for someone else
• Email related to an online dating profile that you did not create
• Email about a password reset or account recovery that you did not initiate
• Email reminder for an appointment or reservation that you did not create
• An email purchase confirmation or receipt for something you did not buy
• Other (please fill in the blank)
• None of the above

Type of misdirected email message	n
An email that contains a personal message meant for someone else	26
Account activation, registration or confirmation email for something you did not sign up for	18
Email related to an online dating profile that you did not create	15
Email from someone you know in real life that was sent to you by mistake	12
Email containing a statement or bill from an account that does not belong to you	11
An email purchase confirmation or receipt for something you did not buy	10
Email about a password reset or account recovery that you did not initiate	9
Email reminder for an appointment or reservation that you did not create	7
Email from a children's website asking for parental consent, for a child that you don't know	5
Other misdirected email	3
None of the above	2

Display if Yes, I have received a misdirected email was selected:
Please briefly describe one example of a “misdirected email” that you have received in the past. Your answer must be at least 150 characters long, which is about three sentences. [fill in the blank]

2.6 Demographics Block

The descriptive statistics for these questions are included in the Respondent Demographics section of this document. The text of the questions are included below for reference.

Please rate your understanding of each term below from None (no understanding) to Full (full understanding): [None, Little, Some, Good, Full] Note: “filtibly” below is a fake word used as an attention check; respondents were excluded in data cleaning phase of analysis if they answered “Good” or “Full” familiarity for this made-up word.

• Wiki
  
• Meme
  
• Phishing
  
• Bookmark
  
• Cache
  
• SSL
  
• AJAX
  
• RSS
  
• Filtibly

What is the last grade or class you completed in school?

• None, or grades 1-8
  
• Some high school
  
• High school graduate or GED certificate
  
• Technical, trade, or vocational school AFTER high school
  
• Some college, no 4-year degree
  
• 4-year college degree
  
• Some postgraduate or professional schooling, no postgraduate degree
  
• Postgraduate or professional degree, including master’s, doctorate, medical or law degree

What is your current employment status?

• Employed full time
  
• Employed part time
  
• Unemployed looking for work
  
• Unemployed not looking for work
  
• Retired
  
• Student
  
• Disabled

What was your total household income before taxes during the past 12 months?

• Less than $25,000
  
• $25,000 to $34,999
  
• $35,000 to $49,999
  
• $50,000 to $74,999
  
• $75,000 to $99,999
  
• $100,000 to $149,999
  
• $150,000 to $199,999
  
• $200,000 or more

Which categories below best describe you? Select all that apply:

• White
  
• Hispanic, Latino or Spanish
  
• Black or African American
  
• Asian
  
• American Indian or Alaska Native
  
• Middle Eastern or North African
  
• Native Hawaiian or Other Pacific Islander
  
• Some Other Race, Ethnicity or Origin (please specify)

Which region of the country do you live in?

• New England - Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont
  
• Middle Atlantic - New Jersey, New York, Pennsylvania
  
• East North Central - Illinois, Indiana, Michigan, Ohio, Wisconsin
  
• West North Central - Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, South Dakota
  
• South Atlantic - Delaware, District of Columbia, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, West Virginia
  
• East South Central - Alabama, Kentucky, Mississippi, Tennessee
  
• West South Central - Arkansas, Louisiana, Oklahoma, Texas
  
• Mountain - Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, Wyoming
  
• Pacific - Alaska, California, Hawaii, Oregon, Washington
  
• Other region (please specify)

3 Interviews Overview

3.1 Participant Descriptives

We conducted 22 semi-structured interviews during October 2017.

Participants were recruited in two ways. 12 participants (7 women and 5 men) were recruited via snowball sampling starting from a study advertisement on Facebook. 10 (5 women and 5 men) came from a paid subject pool consisting of members of the community surrounding Michigan State University. 3 participants were affiliated in some way with the university.

The average age of participants was 36.82 years (minimum: 23, maximum: 63). This study was approved as exempt by the university’s institutional review board.

The interviews ranged from about 14 to 49 minutes (average 28 min) and took place over the phone so that we could recruit a more geographically diverse sample. Interview length varied based on how many examples of misdirected email each participant mentioned.

The data from which the numbers in this document were generated came from participants’ end-of-interview questionnaires, and from things they said during the interview. For example, the number of email accounts they used came from being asked during the interview to list the different email accounts they used, and what they said was subsequently coded and then entered in NVivo as a case attribute.

Participant Age Descriptives
Min	23
Median	34
MaxAge	63
MeanAge	37
StDevAge	11

Interview Duration Descriptives (minutes)
Min	13.8
Median	26.6
Max	49.3
Mean	28.3
StDev	9.7

Email Accounts Per Participant
Min	1.0
Median	3.0
Max	9.0
Mean	3.2
StDev	1.6

3.2 Types of misdirected email per participant

This is a heatmap of the types of misdirected email mentioned, by participant. The categories were determined in an inductive fashion during the interview analysis phase of the project. The horizontal lines delineate four groups of participants with similar patterns of receiving misdirected email, based on the number and diversity of the different types of misdirected email they mentioned in the interviews.

4 Interview Protocol

We used a semi-structured interview method. This means that the interview protocol is a guide, not a script to be followed. The interviews touched on all aspects of the protocol, and the sections of the interview took place roughly in the order presented below. However, the questions were not necessarily asked verbatim, not all of the questions were asked of every participant, and follow-up questions/probes were used extensively.

4.1 Interview Questions

Questions about general email use (warm-up):

I’d like to start the interview by getting to know a little bit about the how you use email in general.

The first thing I would like to do is to have you think about all the different email addresses or accounts that you have, that you can remember, and just list them for me off the top of your head – you don’t need to go look them up.

Which email account(s) do you use most? How long have you had those accounts? How often do you check your email? When was the last time you checked your mail from those accounts?

Tell me about your process the last time you checked your mail before now – where were you? What device did you use? What did you do first? Then what?

Questions about spam:

Next, I have some general questions about spam email. What kinds of emails does the word “spam” bring to mind for you?

If I showed you a bunch of emails, what would you look for to help you decide if a message is spam or not?

Can you think of a specific example of a spam message that you’ve received recently? Describe it. What is it about this message that makes it “spam”?

Questions about receiving misdirected email:

Next, I want to talk about emails that you may receive, that may or may not seem like spam to you. [this depends on what the person said about spam]

The best way I can describe this is, these emails seem like the person who sent them intended for them to go to someone who is not you. They could come from someone you already know, or from a complete stranger, or from a business or website you have never been to. But the key thing is these are legitimate emails, but they just seem like they are meant for someone else.

Can you think of an example where you have received a mail like this? [if they can’t think of an example, review what they said had happened to them, from the screening questionnaire, skip to that part of the interview protocol instead. “You said in the screening questionnaire that… tell me about that…”]

Tell me more about that. Can you find the email now? Describe it. What was it about, who was it from, how long ago? What did you do after you received the email? Etc.

Can you think of another example? Tell me about it. [Keep going if they have lots of examples, up to about 5-6 examples.]

Sometimes people tell us that they’ve received an email asking them to confirm a new account that they don’t remember creating. If this has happened to you, can you tell us about it? Can you find the email now? Describe it. What was it about, who was it from, how long ago? What did you do after you received the email?

Have you ever tried to create an account on a website, but you couldn’t because it said your email address was already being used on another account? Tell me more about that. When did this happen? What website was it? What did you do next?

What other examples can you think of that are similar to what we’ve been talking about? Which ones stand out most in your mind? Tell me more about that, etc.

Why do you think these kinds of things happen? What do you think you could do to prevent this?

Questions about reasons/causes for receiving unwanted email:

Can you think of a time when you sent an email to the wrong person by mistake? Tell me about that time. What was the email about? Who did you mean to send it to? How did it end up going to the wrong person? How did you realize what happened? What happened next?

Have you ever made up an email address, or given out someone else’s email address, when some person or website asked you for an email address? When was the last time this happened? How did you decide what email address to give out? What happened next?

Ask if the participant has any questions:

If the participant has asked any questions about the study or about how email works, now is the time to answer them. If you don’t know the answer, write it down and tell them Dr. Rader will get back to them.

Final Online Questionnaire:

We’re almost done. I just have a very short online questionnaire with a few basic demographic questions for you to fill out. The last page of the questionnaire will ask you for the email address you would like us to use to send you your $15 Amazon.com gift card. As soon as you submit the questionnaire, Dr. Rader will be notified, and she will go to Amazon to purchase and send you the gift card.

4.2 End of Interview Questionnaire

This questionnaire was completed by each participant after the interview, mainly as a way to collect the email address where they wanted the study incentive sent.

1. Participant ID number [Fill in the blank, given to the subject by the interviewer at the end of the interview]
   
2. What is your primary occupation? [Fill in the blank]
   
3. How old are you? [Fill in the blank]
   
4. What is your gender? [Fill in the blank]
   
5. In what city and state do you currently live? [Fill in the blank]
   
6. How did you hear about this study? [Fill in the blank]

After completing the above questions, participants were automatically redirected to a separate survey so their name and email address will not be associated with the questionnaire data.

Thank you for participating! Please enter your name below, along with the email address where you want to receive the $15 Amazon gift card. This information is being gathered separately from the questions you just answered. We are doing this so your survey responses cannot be associated with any information that could identify you.

After you submit this information, the researchers will be notified, and Dr. Emilee Rader will go to Amazon.com to purchase the gift card and send it to the email address you provide. If you do not receive a gift card within 24 hours, please email us at wrong.email@bitlab.cas.msu.edu

Your Name: [Fill in the blank, required]
Your email address: [Fill in the blank, required]