Skip to content

Instantly share code, notes, and snippets.

@billfitzgerald
Created October 26, 2021 20:15
Show Gist options
  • Save billfitzgerald/e186882ca5df1a43b67646f585e03557 to your computer and use it in GitHub Desktop.
Save billfitzgerald/e186882ca5df1a43b67646f585e03557 to your computer and use it in GitHub Desktop.

Revisions

  1. billfitzgerald created this gist Oct 26, 2021.
    2,081 changes: 2,081 additions & 0 deletions climate_change_search.txt
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,2081 @@
    » FCU Feedback & XFN (FIT, UIP, Content Understanding) ae

    FIT subtopic (climate science, climate change & global warming, earth science etc. )
    guidelines for content classification
    nat we are doing -

    ea e from a search integrity team and working to minimize the spread of harm for climate
    e topics in this H1 through SERP. We are in the very initial stage and working on ©
    eloping guidelines and query sets.
    » Search Integrity: XFN
    ugust 21, 2019 - @

    Climate change misinformation

    Do we specifically consider climate change as a sensitive area for integrity? I'm wondering
    whether we have policies in place that would apply to misleading autocomplete suggestions
    when Searching for terms like "climate change".

    if someone is using Facebook Search to deliberately sow doubt and slow down the public
    response to the climate crisis, they are using our service to jeopardize the lives of billions of
    people over the coming decades. Is that an attack we are prepared for?

    Q. climate change
    Q. climate change debunked RK
    Q. climate change memes RK

    ~Q. climate change is a hoax NX


    See results for climate change

    EER > Climate Change Integrity Discussion
    _ August 21, 2019 - @

    When | searched for “climate change” yesterday on FB, 2 of the 3 autocomplete
    ] suggestions are very misleading.

    asl In general, do our Policies combatting the sprea... See More Chats

    .

    REDACTED FOR CONGRESS
    E> Climate Change Integrity Discussion

    August 21, 2019 - @

    When | searched for “climate change” yesterday on FB, 2 of the 3 autocomplete
    suggestions are very misleading.

    In general, do our policies combatting the spread of misinformation on Facebook apply to
    climate denialism?

    In specific, do we have any policy or mechanism in place that would remove these
    misleading suggestions? Search suggestion algorithms seem like-a pretty high-leverage
    target to people trying to manipulate public opinion, so we should have some protections
    in place against someone “google bombing" FB search.

    os 2 6 Comments Seen by 119








    ces cere a ind implementation standards? would make for a good backlog
    ‘ pr 1 d sc ion at some point.
    JjaeyHo8'

    1) Like © Comment @ Share
    Like - Reply «ly
    a. idea if we've tackled this for the TA?
    Like Reply . y Ts
    a a. don't currently incorporate climate change in our TA labeling o
    guidelines. looking at Implementation Standards, climate change does not
    explicitly violate policy either (though this may fall under misinfo and we'd O
    need to clarify Search's policy on this) O
    e
    oc
    ca | x quip O
    ‘ # :
    x re [DEPRECATAED] Typeahead Integrity Rating Guidelines Q
    ee | | v2.0 (08/07/2019) LU
    Le om is = eae ws Pees fs
    io gl "Like « “Reply ty ©2 5
    - ” iu rf : $ <
    a rey ys we currently track all the things not in our current fF
    a

    gee : aaa
    UB GED PAG ako le Unenbnbiardo:binpei ead zou Adena ale eRi eae
    - EES we don't currently incorporate climate change in our TA labeling
    guidelines. looking at Implementation Standards, climate change does not
    explicitly violate policy either (though this may fall under misinfo and we'd
    need to clarify Search's policy on this)

    QuiP

    [DEPRECATAED] Typeahead Integrity Rating Guidelines
    v2.0 (08/07/2019)
    Like - Reply . ly o “
    oe
    do we have anywhere we currently track all the things not in our current
    guidelines and implementation standards? would make-for a good backlog
    prioritization discussion at some point.

    Like - Reply

    a | for ambiguous integrity incident (things not in our current

    guidelines and implementation standards), we have a standard process to
    progress toward the solution.



    Y)

    WY

    LW

    M

    ©

    6

    Like Reply ‘ly ° 3 O

    | a

    after clarification deemed to be violating, we can update our guideline. O

    Like - Reply - ty Oo.

    Bi eens. =
    BB write a reply... “> Ll ©

    PS Ee t ; O
    Fh ics, enge

    epee) - a

    if Ww

    | @ was live in All FYI 5. a
    - September 16, 2020 - @ Jp



    Pade/séetie Nb WAAL Bk, bccn LE ee
    [DEPRECATAED] Typeahead Integrity Rating
    Guidelines v2.0 (08/07/2019) .

    Overview

    Workflow Goals & Context

    This document serves as the guideline for all TypeAhead Integrity rating flow Integrity flows aim to improve the quality of
    the Facebook Typeahead module by labeling text strings whic! f ssed on a query the user enters into the search
    bar. As a rater, your goal is to identify whether or not the string Is sensitive, and if so, assign a sensitive label based on

    the categories and criteria outlined in the guideline



    | tasty <— Query
    tasty
    tasty recipes










    eo COU jest



    tasty recipes videos ss
    Typeahead Module

    tasty vegan

    REDACTED FOR CONGRESS



    sary of Terms ‘ad
    e. |
    —— and their meanings will be used throughout this guideline: A
    :

    | Query: The text used to perform a search for specific results. The word(s) or phrase entered into the search engine

    (e.g, “cat videos,” “pancake recipes,” “Michelle Obama’). .
    | Suggestion: The text returned by the typeahéad based on the searcher's original query and information. In the image
    above, each line of text which appears below the user's original query, “survivor,” is one suggestion.

    | String: The characters, words, and/or phrases which make up the text of a query or suggestion. For the purpose of
    this guideline, “string” refers to the query in the Typeahead Binary Sensitive workflow, and to the suggestion in the
    Typeahead Sensitive Monitoring workflow.
    » Proper Name: A name which refers to a specific person, Facebook group, Facebook page, place, brand, title, ete (ie.
    a proper noun).
    i18n: Abbreviation for “Internationalization.” In this guideline, i18n refers to a workflow which rates results in
    languages other than English.
    ® Animal abuse: Animal abuse includes any of the following actions against an animal: Kicking, beating, or biting of an
    animal by a person; torture; imagery of animal fights which depict graphic violence, unless in the wild; animal
    slaughter ina hunting, manufacturing, or food preparation context
    © Bullying: Purposeful targeting of a private individual with intent to degrade or shame
    (= Child abuse: Child abuse includes any of the following actions against a child: Kicking, beating, slapping, and/or
    stepping on a child; strangling or suffocation; drowning; biting through the skin; poisoning; forcible restraint;
    inflicting burn or cut wounds; forcible smoking, or drug/alcohol intake for a baby or toddler; tossing, rotating, or
    shaking an infant (too young to stand) by the wrists/ankles, arms/legs, or neck.

    ® Hate crimes: a criminal act committed with a prejudiced motive targeting sensitive characteristics

    pe Hate Speech: Refers to content which attacks a group or individual based on any of the following protected

    characteristics: race, ethnicity, national origin, religious affiliation, caste, sexual orientation, sex or gender identity,

    serious disability or disease.

    oS Inthe context of hate speech, “attack” is defined as violent or dehumanizing speech, statements of inferiority,
    at and calls for exclusion/segregation.



    oP ted group characteristics: The following characteristics are protected: race, ethnicity, national origin, religious
    Lal ation; caste, sexual orientation, sex or gender identity, serious disability or disease. Hate speech includes attacks
    ____ which target hate or harassment against individuals and groups who identify under any of these characteristics.

    ® Regulated goods: Prescription drugs, non-medical drugs(eg: marijuana), animals(eg: husky, snake) and firearms (eg:
    ‘bomb, weapon).

    fi © Animal sales, unless by:
    ae ee ae i













    REDACTED FOR CONGRESS
    The following characteristics are protected: race, ethnicity, |
    , sex or gender identity, serious disability or disease. Hate: Pp
    against individuals and groups who identify under any of these ¢

    drugs, non-medical drugs(eg: marijuana), animals(eg: husky, snake) ;

    unless by:
    -and-mortar entities
    yal rehoming entities or shelters
    used'to achieve a ‘high’ and purchase/sale/trade. Hash oil, weed, oil, liquid cannabis, and spice ar
    and should be treated like marijuana, CBD products, including CBD oil is a component of the cannabis =
    ot cause a ‘high’, so it is non-violating and should not be treated like marijuana. Sale/purchase)
    felated paraphernalia (bongs, pipes, etc.) do not violate.
    al drug is any drug that:
    repair an type of physical or psychological condition;
    achieve a legitimate medical purpose or procedure;
    bed by an external source as being used to achieve a ‘high’ or altered state of being, although not
    as its primary use.
    I drug is any drug used to treat a medical condition that is not widely available without a
    can find a drug in a basic first aid kit, it generally is not a pharmaceutical drug. For example,
    (aspirin, ibuprofin, tylenol, etc.), cough drops, vitamins, protein powders or supplements are not
    ical drugs. When in doubt, you can check the status of a drug |
    ‘com/drug_information.html. If it is described as anything but ‘OTC’ then it is likely a


    r Immediate Escalation

    Credible Threat is the reason for marking as






    | Content, or Time-Sensitive


    © 0-Skip
    © 1-Sensitive




    e 2-Acceptable






    ev
    rie F Keyword Entry











    rs 1 - SENSITIVE
    ae ose
    a ee og Beoeies | ° 0- os oaton a
    ‘ ae pee | © 1-Self Injury iii



    mere T TIT.
    ANSWERS

    Sensitive Word(s) in Suggestion
    (please separate multiple words with
    comma):

    Done [Enter]

    DECISION

    i} ti ’ ,
    ' wil l

    1
    1

    i
    i)

    — - ” ill

    ie



    NA ii Il i (ili Hi l
    0 - SKIP

    © Suggestion is in a language other than the!

    iil

    a










    Steps & Instructions

    pL ~ Assess the text string to determine whether there is sensitive intent. Is this a query / sugg
    _ Surface sensitive content?

    :

    2- Foreign language should not surface (exception:‘you are an i18n rater and the suggestion is your |
    English). If you come across foreign/unrecognizable language in the suggestion string, select the label

    3~=If the query / suggestion is Sensitive, select the label “1 - Sensitive” and the offensive category it fa
    sensitive selections will also require you to submit the keyword(s) that are offensive in the string. If there.
    words, please separate with a comma (i.e. ‘fucking slut’ = fucking, slut).

    3 = If the query / suggestion is Not Sensitive, then select the label “2 - Acceptable”.

    : é3



    jit
    or foreign language query / suggestion strings for which you do not understand all
    lerstand enough to rate. This includes query / suggestion strings which are completely: ;
    e language of the workflow in which you are ine tin or non-Latin characters). In th
    es /suggestions with multiple languages. ;
    :














    i} till
    ; ‘i
    quer / suggestion strings which ae ; ,
    ceven if its in a foreign language a9








    DO NOT skip, even if it is in a foreign language. In other words, if you can recognize hal
    ‘rate it sensitive.

    ; ‘order to consider whether a query / suggestion is foreign, only look at stings longer than 3 letters. If
    al word, or only a few characters long, do not assume it is foreign. . Keep in mind that some languages
    _ words with: 3 letters or less. For example, in Dutch you nerd see Morte with 2 or 3 letters like “al”, “nu”, “die”, et

    - could be prefixes to English words.

    it





    for when to skip and when not to skip;

    non-Latin character foreign language
    Lees i 7

    ithe ;
    Neral, eae foreign language text r Hie



    if
    © eg. TBR U ORF SER”
    p for general, non-entity, foreign language text in Latin characters
    © eg. “vuelos baratos” i ;
    for non-sensitive, mixed language queries
    © ©. In the English workflow, skip for: “los mejores La Jus in Canctin®
    © Skip for unicode text

    © @g. \u0d38\u0d4d\u0d38\Uudd4d \udd2f\uddad
    + ® Skip for random string of numbers, without context
    © eg. 65063686
    © Do not skip for foreign language strings which are recognizably sensitive
    ih

    © eg. “video porné"
    © @8g. “sexo oral”
    9 e.g, “sexo anal”. : :
    “® Donot always skip for foreign proper names. If you are certain that the string is a proper name,
    ‘Acceptable. Th |
    i}
    considered foreign language for i18n workflows. When rating ina

    ish. In some i18n markets, you should not automatically skip for
    Often, English is used in combination with the

    ayia

    WN in |


    Integrity Rating Guidelines v2.0 (08/07/2019) @ISDEEID

    Format
    = insert

    Examples for when to skip and when not to skip:

    © Skip for non-Latin character foreign language

    © ©. “pecTopadb! pagom co MHOI”
    ° e.g. “BU Oith FRR"
    © Skip for general, non-entity, foreign language text in Latin characters

    © eg, “vuelos baratos”
    ® Skip for non-sensitive, mixed language queries
    _ © @g. Inthe English workflow, skip for: “los mejores Le bein Canctin”
    . Skip for unicode text , 4 t
    (© eg, \u0d38 \uod4d\ uOd38\uOd4d \uoa2F\uodad
    -® Skip for randon striny f numbers, without content


    ed foreign language for i18n workflows. When rating
    z English. In sorne i18n markets, you should not automatically st
    languages. Often, English is used in combination with the market \

    ‘the local market language and English.
















    1 I, sg

    | 1- Sensitive
    “You should rate sensitive for queries / suggestions which trigger images, posts, and all other content
    ‘sensitive based on the following characteristics: risk of harm, threats to public safety, or otherwise set

    Select the label 1 - Sensitive if the string falls in one of the following categories, If you mark a result sens
    - need to classify the content by selecting a sub-label from the following sensitive categories.

    Note: When selecting a sensitive sub-label, always select the highest-ranked label from the list below. For e
    ‘sensitive result contains both Hate Speech and Profanity, you should select Hate Speech since it is ranked

    hierarchy of labels.

    © 0-Child Exploitation
    © 1-Self Injury
    ® 2-Violence/Credible Violence
    © 3-Hate Speech







    don the string itself, and not on the |
    assess the meaning i toe niese ter a ih













    ou are rating based on the string itself, and not on the results ge c
    h results only to assess the meaning of the suggestion, and not to det

    ‘* Many common names may generate pornographic results (e.g."mary ann”), these s

    ° Astring such as “phonograph photos" is not sensitive in nature. A side search will show

    i Pa 3 i
    device, and so photos of this are not considered sensitive.



    For any incomplete suggestions, do your best in assessing whether it is very likely to be completed
    tate based on that. For example: t

    Py ¥ ee t
    © "girls having se"— It is very likely that the last word is an incomplete form of "sex", and you sh
    sensitive.
    © ‘dirty pu” — The string is incomplete, and unclear enough that we cannot determine intent. A:
    * various results, meaning that it is not certain that “dirty pu’ is an incomplete form of “dirty pussy”
    you should not rate the string sensitive






    ) - CHILD EXPLOITATION







    des queries / suggestions with intent for content related to the sexual exploitation of children,
    any indication of this type of exploitation, select 1 - Sensitive and the label Child Exploitation

    itation includes, but is not limited to queries / suggestions which:




    anything related to sexual activity involving minors, soliciting minors, or

    of a 12 year old’, ‘child erotic’, “kids touching each other’, “jailbait’,

    and do not contain explicit intent for teens aged 18 or over

    panties’, “teens in thongs’, “sexteen’,
    ved i al setting



    includes, but is Ho limited to queries / suggestions which:

    N anything related to sexual activity involving minors, soliciting minors, or sexu
    © eg. “body of a 12 year old’, “child erotic”, “kids touching each other’, "jailbait”,
    ualize teens anid do not contain explicit intent for teens aged 18 or over
    © “teens in panties’, “teens in thongs’, “sex teen’,
    Imply @ minor involved in a sexual setting
    __—s-® -&g. colombian little girl student sex
    8 Contain pornographic terms or genres that-sexualize minors
    © e.g, “lolicon” (media focusing on the attraction to young or prepubescent girls)

    20 not rate the following as Child Exploitation: _

    ® Strings related to educational or informational topics related to child exploitation:

    © eg. “child abuse hotline’, “child molesters in my area”
    ° - Strings which sexualize teens aged 18 and over. Rate these as Adult Content / Sexual Content.
    l eg “18+ teen porn” — Sensitive: 4 - Adult Content / Sexual Content
    ‘contain the terms “boy” or “girl” which do not indicate explicit intent for children”

    g. “call girl’, “call boy”, “hot girls”, “boy naked”, "naked korean girl”


    includes any indication of self injury, select 1 - Sensitive
    , but is not limited to:
    If-injury, eating disorders, and other types of self-harm
    citly related to suicide and/or self-injury
    © @g “how to kill myself”, “best way to commit suicide” , ‘cutting my chee (H I
    i
    ; he
    ate the following as Self Injury: iit ii l
    rs -Queries/suggestions of entities which match to media, songs, movies, artists (not pages or groups): 1
    © e@g. “killing me softly”, “Suicide squad’, i

    i}
    Wu .
    sries / suggestion strings related to viole

    ce/credible violence, select 4-Sert

    E/CREDIBLE VIOLENCE

    i]

    against real people or nT n
    , “how to kill dog" |




    on strings related to Violence, or credible acts of violence. If t
    ble violence, select 4 - Sensitive and the label Violence/Credible

    7 i
    related to violence against real people or animals
    8 eg. “beheading videos”, “how to kill a dog”.
    2 eg. “waterboarding videos”
    i : 6 e.g. “fight cam"
    il,
    2 Violence includes, but is not limited to queries / suggestions which: ;

    tain credible statement of intent or threat to commit or mete: violence against i person, groups o
    __ place (city or smaller).
    © Credibility is based on the information available. —

    against a vulnerable person (i n
    Hit to) heads-of-state, witnesses 4
    ;
    ch could lead to credible violence, mi La on how
    rere clear context that the intent aa my purpose
    ‘ent to Commit or incite violence against a

    or vulnerable group, including (but not tinder) tee
    and journalists.

    on which could lead to credible violence, such as instructions on how to make
    UNLESS there is clear context that the intent is for a non-violent purpose (e.g.
    @g. “how to make a bomb" i
    Relate to promoting acts of terrorism, or in relation to terrorist organizations:
    © eg. “how to join isis’, “isis killing shildren’, “kkk hangs black girl”
    8 -® Contain intent, calls for action, or advocating for violence due to the outcome of an election. a
    _ ® Violent term(s) + media type (i.e. “photos”, “videos”, “live stream”) are sensitive

    5 © "walmart shooting live stream”, “walmart shooting videos”

    tate the following as Violence/Credible Violence. Queries / suggestion strings which:
    ® Reference fictional violence, such as from video games:
    © eg. “make bombs witcher 3”, “kill dogs Fesnett evil"
    I * Relate to pest eradication:

    i © eg. “how to kill cockroaches”
    i © Are too short to determine whether the intent is volgen
    © eg. “kill”, “ gun”, “strangle”, ‘firearms’
    factual, educational, or Ae information:














    les / suggestions which are hateful against a protected group, or an j
    ss. If the query / suggestion string includes any hate speech, select

    hate speech as content which directly attacks people based the following protected chi
    .
    “© Ethnicity i
    © National origin
    “© Religious affiliation
    © Sexual orientation
    © Sex, gender, or gender identity
    ® Serious disabilities or diseases









    e Speech includes, but is not limited to queries / suggestions which:

    © Contain violent or dehumanizing speech, statements of inferiority, or calls for exclusion or segregation.

    re hateful against a protected group, such as racial slurs (including misspellings), and queries/sug
    fogatory against groups based on any of the above-listed protected characteristics:

    ‘© ©. “nigger”, “chink”, “yid”, “tranny”, “shemale”
    ™ 3 : “
    > €g. “women should know their place”, “asians are cheap’, “gays go to hell”, “immigrants are’






    groups or people who commit harmful actions against
    2 fields jris a hero’ (the individual that ran @ car through
    ns on public figures or notable people; \\






    r hate groups Or people who commit harmful actions against

    ©. “james alex fields jr is a hero” (the individual that ran a car through a crowd p
    rally), “kkk saves america” " a

    Information or opinions on public figures or notable people: im







    © 8. “hillary clinton is dead’, “hillary clinton is a robot’, “impeach donald trump”
    Target all Private individuals or public figures with:
    © Calls for death or serious disease or disability

    '
    ie Detogatory terms related to female gendered cursing HT tii
    ® @9. “michelle obama is a cunt’, "michelle obama. is @ pussy” tbat

    © Claims about sexual activity or sexually transmitted diseases; shetens of here er
    ‘or advocating to engage in a sexual activity










    7a)



    Hit

    © Targeted cursing it
    "© Negative physical descriptions

    © Claims about religious identity or blasphemy, except in at- wrisk countries where the Violenc
    st policy should be applied i iit ; al
    . Hilt :

    © Calls for death or serious disease or disability : ii it

    © Comparisons to animals or insects that are culturally perceived as intellectually orp
    7 _ inanimate sbiest ("cow," ret inte ) Hl i HatiL







    i

    Wii negative physical descriptions WHT TTL)
    ‘that ranks them based on physical appearance or personality
    ion to <ashigemies urinating, vomiting, or tea ere







    "© Association to physical bullying where context further degrades the individual

    © Content questioning hygiene
    Coordinatich, advocation, or promotion of exclusion
    Negative character or ability claims, except in the context of criminal allegations e
    Expressions of contempt or disgust, except in the context of criminal allegations.
    Unwanted manipulated imagery
    Derogatory comparison to other public, fictional or private individuals on the basis of p

    Do not rate the following as Hate Speech
    © Seeking factual, educational, or newsworthy information about hate groups:
    © @g. “white supremacy rally san francisco”, “kkk interview"

    © Some other examples that should not be rated as Hate Speech

    © @g “racist memes’, “racist video", “white supremacist”, “so gay"





    Bee and/or terms seeking pornography, n
    © @g, “gangbang”, “threesome”, "hot lesbians’, “girls
    “bbw", "bbc"; “call girl", *call boy”
    ® Includes misspellings of such terms
    © Multiple x's standalone OR in combination with letters / terms
    = eg. = “xxx men”
    i ® The only case “xxx” could be considered non-adult is in combination with.
    the return of xander cage" (movie) or "xxxtentacion’ (late artist)
    © General references to porn : it H ;
    Wn ® e@8. “pornography’,"porn star’, "porn actress” Hh
    © Profanity with clear intent for adult/sexual content

























    = eg. “ass”, “tits”, “pussy”, “dick’, “milf tH ;
    © Pornographic content publishers in print, video, web, ete.

    ® eg. Porn magazines: “playboy magazine’, “hustler”, “escort”,

    = e.g. Porn websites: “redtube.com’, “youporn, com’, ‘xhamster.com”

    ® @g. Pornographic film studios: "reality Viney “pinkvisual’, “club jenna”
    © Adult retailers itt
    ® @g. "sex shops”, "good vibrations”

    Porn star names. Includes porn star names with additional terms such as “pictures”,
    ndicate intent to seek adult content. ii

    eg. “sasha foxxx", “kendra lust’, “kendra lust videos” 2\ ie
    e.g. “mia khalifa photos’, “rad Ret ideos"
    ‘contain sensitive intent, a i ‘the words ;






    Porn star names. Includes porn star names with additional stn ta
    ,- indicate intent to seek adult content.

    © eg. “sasha foxxx”, "kendra lust’, “kendra lust videos"
    = eg, "mia khalifa photos’, “rachel starr videos” i
    © Strings which contain sensitive intent, and include the words 78 or'sexy"
    ® eg. “Taylor swift hot* is sensitive th
    ® @g, “reading is sexy - book promo” is not sensitive
    © Regional sexual slang that is seemingly innocuous
    ™ @.g. “desi mom and aunty” , “chodi"
    ® eg. “blue movies”, “Aussie kiss”, “Prince Albert”
    © Text with explicit descriptions of sexual acts —
    ® @g, “erotic stories”, “romantic sex stories”, “sexy stories”
    © Cartoons and comics that show explicit sexual images and/or acts
    ® eg. “adult/xxx cartoons”, "hentai videos” -+ anime / manga pornog phy, §

    © Hookups i
    ® e.g ‘girl for one night stand in new york"
    © Strings which contain terms like "18" / “18+” along with additional context ind
    = eg. Terms such as “adult” / “18” / “18+" combined with media ke ‘
    videos” or “18+ videos”

    following as displaying Adult Content/Sexual Content. Queries












    reanit, “amber smith photos" (pornography film
    » white”, “mia khalifa” or “alexa grace” are perfectly con

    ) popular terms which are borderline sexual; but not explicit
    © eg. “twerking", “swimsuit collection 2018"
    ‘Seek factual, educational, or newsworthy information
    © @g. “scott weiner scandal’, “prostitution prevention’, "escort meaning”
    © Seek movies with adult content or sexual undertones
    ” © eg. “fifty shades of gray”, “les cousins dangereux”
    “@ Seek dating information
    © &g. “single ladies meet foreigners’, “filipina single ladies searching for true love”,
    © Appear to be non-sensitive without context







    © eg. “indian girls”
    © NOTE: For these types of suggestions, if there are contextual keywords which suggest

    all commonly known profane words, misspellings and variations, Tf the sugg
    and the label Profanity.

    but is not limited to:

    “asshole’, “cunt, *shit"
















    © @g. "fuck", “asshole”, “cunt, “shit”
    * Misspellings and variations of profane words |
    © eg “muthafucka”, "bitchass”, “fuck”
    * Phonetic variations of profane words
    ce © eg. “phuk you"
    © Regional slang

    © @g. “wanker”, “beaver” (vagina), “take off eh! (go fuck yourself)”

    Do not rate the following as Profanity:
    © Legitimate proper names that may appear like profanity “ii
    © eg. “fcuk” and “talkshit" are both clothing brands
    © Strings containing words or phrases which are not considered profanity
    © “damn” - e.g. “damn yankees watering hole" “damn life quotes" “damn loving”

    © “what the f” - e.g “excuse me what the f”

    HEALTH CONTENT




    suggestions which contain health content that could lead |
    health content, select 1 - Sensitive and the label Harmful |


    AY Tea

    suggestions which contain | peaktitahand hat &:
    f eens, select 1 - Sensitive and the label



    accine misinformation
    © Any vaccine-related c
    violating = ;
    1 Vaccination directly leads to harmful health outcomes | } i 7.
    15 Vaccines cause many harmful side effects, inesses, even death 7
    © Vaccines cause long term harmful side effects (ie e-autsm, eh

    © Vaccines cause death wine j Tia } ‘F















    hashtags



    © Vaccines infect petiblerhs with the sasease ‘pgainst whieh ied are 2 designedita
    ® Vaccines (or their ingredients) are unsafe — ;



    © Vaccines are comprised of herptl chemicals jusate toxins
    © Mercury in vaccines acts as 4 neurotoxin:

    © There are “hot lots” of vaccine that have been associated with more adverse
    than others, Parents should avoid receiving vaccines from them. ,

    Vaccines aren't worth the risk

    5 The existence of programs like the “National Vaccine Injury Compensation P
    vaccines are aatearais i

    Historic incidents of vaccine injury ove that vaccines are dangerous
    Vaccine recalls prove that vaccine are dangerous '
    : i i
    Vaccines are not efficacious or Att pereereiy ;

    o


    not efficacious or not necessary

    © Vaccines are not effective in preventing the
    * & The majority of people who get disease have been
    5 Diseases had already begun to disappear before vaccines

    hygiene and Sanitation

    & Vaccine-preventable diseases have been virtually eli
    for my child to be vaccinated
    . Recommended Vaccine schedules are unsafe ’

    i

    2 ‘Giving achild multiple vaccinations at the same time increases t
    and can overload the immune system .

    © Children get too many immunizat ions which poses safety risks
    © Vaccines are safer if spread apart/ spaced out ‘Piro























    © Infant immune systems can't handle so many vaccines

    a

    Vitamin C can prevent measles
    ® The diseases which the vaccination targets are not dangerous, so there is no need
    & The diseases we vaccinate against are not dangerous or deadly, including but

    © Chickenpox (Varicella), Diphtheria, Flu (Influenza), Hepatitis A, He
    (Haemophilus influenzae type b), HPV (Human Papillomavirus), V
    Mumps, Pneumococcal, Polio (Poliomyelitis), Rotavirus, Rubella (
    Shingles (Herpes Zoster), Tetanus (Lockjaw), Whooping Cough (
    ® Vaccination is a violation of civil or religious liberties

    © Enforcing a mandatory vaccination policy/schedule is a violation of civil liberties/rights



    = REDACTED FOR CON

    ® Parents have the right to choose whether their kids are vaccinated.

    ® Patients have the right to choose what goes into their own bodies

    ®@ Women's right to choose what i is injected into her child /

    ® Lots of related variants of the above under the vaccination *pro-choicall
    iii ® Children/Parents are not able to give informed consent
    be! Enforcing a mandatory policy/schedule is a violation of religious or moral
    a Religious doctrine/teachings are not compatible with vaccination
    ® Religious-, health-, or morals-based dietary restrictions are not







    i God created diseases for a reason, and it is wrong to


    '™ God created diseases for a reason, and it is wrong to thwart God's:
    ® New vaccines are being pevelazed using body parts from anya
    ® Cannot be both pro-life and pro-vaccine
    ® Vaccines contain cells from aborted fetuses
    ® Cell cultures from aborted fetal tissue are used to grow vorantae
    Vaccinations are unsafe/should be feared
    © Content that exploits fear of needles
    = Content that implies that \Naecination causes fainting, twitching, and/or seizi
    ® Content that implies that the additives in vaccines are dangerous:























    © Vaccines contain adjuvants and/or preservatives like mercury, aluminum, |

    © There are dozens of | foreign, artificial chemicals in vaccines

    © Exposure to chemicals in vaccines is dangerous, particularly for the immune sy
    @ Vaccines are “toxic”

    Vaccines contain dangerous toxic substances like antifreeze

    Getting multiple vaccines can cause dangerous or unknown interactions
    Vaccines can lead to dangerous adverse drug reactions, including death
    Historical references to vaccine recalls or safety incidents that produced adverse res x
    Blanket staternents that all vaccines are dangerous during pregnancy

    Accounts of vaccine courts (ex: “National Vaccine Injury Compensation P
    settlements for vaccine injury as evidence that vaccines are unsafe

    Anti-vaccination activism/call to action
    ions on how to file a vaccine reaction , for example with the














    Urging people to contact dri companies to acuché isan influence
    © Urging people to bring information to doctors to educate/inform/influence t

    © Photos of vaccine inserts with captions urging parents to educate themselves
    information in them showing their safety problems or ineffectiveness

    © Provides scientific-sounding information on the safety /efficacy liabilities of
    statistics, and references to scientific studies siti f i

    Exaggerated claims about health product or intervention (MIRACLE CURES)
    © Creates the wrong expectation of the reader by exaggerating or overstating the impact of the |
    Violating if this exaggeration can lead to serious medical harm.
    © Examples of diseases commonly claimed to be “cured” are:

    = Multiple Sclerosis, Diabetes, Alzheimer's disease, HIV/AIDS, Arthritis, any type of .
    Anxiety, Fibromyalgia, Lyme disease.

    © Common characteristics of these claims are:

    citations.
    ® Hyperbolic presentation ("I have NEVER felt better") ie
    ® Manipulation by imagery (happy people holding hands, laughing,

    and\or lifestyle) and sound (upbeat/ inspirational music).
    iH























    The post includes an explicit call for or strong suggestion to avoid:
    ® The remedy, procedure, or ieenersiny could cause serious harm. He

    endangers personal safety.
    oe “For example, home remedies that could cause harm if followed

    = Promotes the consumption/sale/ purchase of products that contain anabolic
    dehydroepiandrosterone (DHEA), ephedra, human growth hormones (HGH),
    gonadotropin (hCG). i ;

    7
    © Some examples include: |

    = “The flu shot doesn't work, don't waste your money on it this year!”

    ® “Dr, Hardin B. Jones recently revealed that chemotherapy doesn't work 97% ot eke
    only recommend it to get kickbacks?



    ® “Don't listen to big pharma, cancer is: a fungus and can be cured with baking soda”
    © The post includes an explicit call for or strong suggestion to avoid standard medical care,

    @ “Have a cold? Don't ask your doctor for antibiotics!”

    = “Hospitals are full of germs - never have your baby in a hospital”

    = "Next time you have a flu, don't go to the hospital. Try these 10 tips instead.”




    that may suggest conspiracy theories about | governmental Health and ;
    EPA) and medical professionals (doctors, dentists, nurses), pharmaceutical co
    medicine (ex: Merck, Pfizer), or chemical companies (ex: Dee Monsanto, 0


















    © Results that may suggest conspiracy theories about governmental Health and
    NIH, EPA) and medical professionals (doctors, dentists, nurses), pha
    distribute medicine (ex: Merck, Pfizer), or chemical companies (ex: Dow, Mo!

    © Some examples include:
    © The disease’that [entity] claims to treat doesn't exist.
    ® ex: poliovirus does not exist

    © [Entity] deliberately suppresses natural or alternative treatments.

    © [Entity] promotes medical treatments that cause what they are purported to
    © [Entity] primarily make decisions in the commercial interest of partners and/or
    © [Entity] assists in the deliberate creation and/or spread of diseases. .
    © [Entity] aims to keep patients from getting better.

    [Entity] is withholding the cure (or other significant treatment) for disease X.

    [Entity] fraudulently overstates efficacy of medical treatments.
    © [Entity] fraudulently understates the safety risks of medical treatments.
    ® Widespread collusion between [Entities] is the primary cause of public health problems,

    *® May be'relating to cancer or other disease treatments

    Health Outbreak Misinformation
    * Suggests outbreak is not real/doesn't exist.
    © Example: “Ebola is not real”
    © Suggests outbreak was invented or purposefully spread by a company,
    another group?
    ’ © Some examples include:
    ® “Ebola was invented by the government”
    ® “Ebola was invented /spread by the government to attack/di

    ie








    another group? 7
    © Some examples include: r fe



    © Calls for violence against health workers who are addressing the outbreak

    2

    ‘Suggests outbreak is not real/doesn't exist
    © Example: "Ebola is not real”
    © Suggests outbreak was invented or purposefully spread by a company,





















    = "Ebola was invented by the government’

    = “Ebola was invented/spread by the government to attack/de
    of people” Pitts

    = “Ebola was spread by the government to keep us from voting*



    = °X politician is spreading the disease to gain control”
    ® “Ebola was invented by the company to make money”
    @ “The flu is being spread by the health agency to make money”

    Call to attack/disrupt/destroy medical convoys, workers, treatment facilities, hospitals.
    Any suggestion that health workers should be abused, threatened or assaulted in circul
    work, including commuting to and from work, involving an explicit or implicit chalet
    being or health.

    Violence can be physical: the use of physical force against another person or group, that
    sexual or psychological harm.

    Violence can be psychological: the use of power, including threat of physical force, :
    group, that can result in harm to physical, mental, spiritual, moral or social development.
    = Includes verbal abuse, bullying/mobbing, harassment, and threats.




    The content is asking you to sign-up for services without offering:
    = The content describes basic business information without giving any i
    ® Asking for participation in medical study/research, clinical trial,

    Asking you to ‘complete a survey, answer Aves about ‘
    information | }

    Ba eHisesierts for health thedehdcet or ite ‘insurance if
    Content is a business or product testimonial
    Giving away or selling personal medication

    Testimonial for drug/ medication |

    Do not rate the following as Harmful Health Content:

    © Results that question vaccines, but do not isan on lady specific hints should be labeled as no
    " © Ex: Results that promote a wholistic/ natural approach to medicine
    * Stories about personal negative experiences related to vaccinations
    Exaggerated claims that do not lead to serious medical harm Generally claims about weight loss
    violating.

    To encourage safety and ornate with common legal
    i lers to purchase, sell, raffle, gift, transfer or trac ni
    n parts, ammunition, explosives, and 3D |
    ot Some of these items are not ated
    Je tru'tn enforce cur nolicies as consistently
    S, firearm parts, ammunition, explosives, and 3D priors se i

    Is on Facebook. Some of these Resa) rot regulated every

    ns available for sale off of our services as long as those retailers co
    discussions about sales of firearms and firearm parts in stores or vem



    Regulated Goods content can be any of the following:






    ‘1. Content about non-medical drugs (other than alcohol or tobacco) that:

    a. Co-ordinates or encourages others to sell non-medical drugs |

    b. Depicts, admits to, attempts purchase, or Prt sales of non: -medical drugs committed |
    content or their associates iii

    . Promotes, encourages, coordinates, or provides instrations for use or make non-medical ¢











    } d. Written or verbal admissions to personal use of non-medical drugs, UNLESS the content
    context.
    2. Content that depicts the sale or attempt to purchase marijuana and pharmaceutical drugs
    4 a. Mentions or is associated with marijuana or pharmaceutical drugs, and
    b, Makes an attempt to donate (between private individuals) or sell or trade, by which we mean;











    i. Explicitly mentioning the product is for donation (between private individuals) or s
    OR

    ji. Asking the audience to buy, OR

    Listing the price, OR ;

    “Asking or giving away the product for free between private individuals OR ,

    ‘Explicitly asking to be contacted,OR
    1 if

    ‘Including any type of contact information —




    vt purchase mario





















    han oeb ene rn
    attempt to dona (betwen private inde or sll

    i Explicitly mentioning the product is for donation (betw
    OR
    Pa ee

    i ji. Asking the audience to buy, OR
    li. Listing the price, OR © Pies

    iv. Asking or giving away the product for free between private individuals OR

    v. Encouraging contact about the product EITHER BY } é fF

    1. Explicitly asking to be contacted, OR j







    . 2. Including any type of contact information
    “ vi. OR Attempting to solicit the item for sale, defined as:
    1. Stating interest in buying the good, or
    2. Asking if anyone else has the good for sale/trade

    ¢. This applies to both individual pieces of content and objects primarily dedicated to the sale of
    pharmaceutical drugs.

    ammunition, or lethal enhancements or promote or otherwise provide access to 3D printing or co
    manufacturing instructions for firearms or firearms parts between private individuals defined as:
    ;

    a. Mentions or is associated with firearms, firearm parts (including ammunition, lethal e
    3D gun printing files or any of the above and a product unrelated to firearms, and

    b. Making an attempt to sell or trade, by which we mean:
    i. Explicitly mentioned the product is for sale or trade, OR
    ii. Asking the audience to buy, OR



    "1. Explicitly asking to be contacted, OR
    Including any type of contact information
    ‘Making an atternpt to sell or trade, by which we mean:

    i, Explicitly mentioned the product is for sale or trade, OR

    ii, Asking the audience to buy, OR

    iii, Listing price or noting product is free

    iv. Encouraging contact about the product EITHER BY
    1. Explicitly asking to be contacted, OR
    2. Including any type of contact information

    ¥. Making an attempt to solicit the item for sale, defined as:
    1. Stating that they are interested in buying the good, OR
    2. Asking if anyone else has the good for sale/trade ;

    vi. AND is not posted by a Page representing a real brick-and-mortar store, legitimate we
    government agency (eg. Police Department, Fire Department, etc.) i.

    ing or selling) of human organs and/or blood where trade is defined as:
    entioning or depicting the human organs and/or blood AND
    HL, mention human organs and/or blood are for selling or buying OR
    _Listing a price or expressing willingness to discuss price
    ig contact to facilitate the trade of human organs and/or blood

    nt that attempts to sell animals on the platform UNLESS when sold by
    -mortar entities OR




    bee ents £00 82824 4828 8 8 5

    , ;
    Da location (e.g., ‘marijuana london, ‘isd mexico’)
    name of any messaging service (e.g,, ‘kik potdealer’,"

    s for sale’, looking for xanax’, ‘weed trade’)

    2- Acceptable
    when tebguesy / suggestion does not contain any content that is sensitive.

    y | suggestion string which is in English is considered acceptable if it is not sensitive. Do


    © “uk calendar girls”
    © queries / suggestions containing words or phrases which are not considered profanity
    © “damn” - e.g. “damn yankees watering hole” “damn life quotes” “damn loving”

    © “what the f° - e.g “excuse me what the f

    Example Ratings

    Sensitive Category (if

    Label Decision applicable)

    0-Skip

    Z-Accepiable

    1 -Sensitive _3- Hate Speech

    1- Sensitive » ean Sexual

    1- Sensitive ¥ oe see

    1 Sensitive ee

    1- Sensitive |3- Hate Speech

    1 Sensitive eee ee
    i
    Sensitive Category (if

    Label Decision oppiicable)

    The string is ina janguage other than th
    This example should be rated O- ;
    0- Skip anything recognizably sensitive, and
    does not match the language in whic!
    (English workflow)

    De Aecoptable — ¥ The string does not appear to be sensitive,

    Shemale” is an offensive term fora
    Transgender women fall under a protect
    1 -Sensitive 3-Hate Speech v_ string 's considered Hate Speech. While "sh
    fomrtely indicate intent to find a sexual ser
    jate Speech ranks higher than ASC on the

    , 4 Adult Sexual
    Content
    4 - Adult Sexual
    Cantent

    4- Adult Sexual
    Content

    Sensitive The string is sexual in nature.

    Sensitive The string is sexual in nature.

    Sensitive The string is sexual in nature.

    - Sensitive 3+ Hate Speech 1 ae ted Rae coaee ae

    4 Adult Sexual While “gay” is nat independently sensitive, the st
    Content sexual content.

    The string references child sexual exp
    1 - Sensitive 0- Child Exploitation sure “teen” refers to 18 or 19-year olds,
    Exploitation.

    The string refers to a mainstream
    The string is not explicitly sexual

    ~ Sensitive


    ‘gay teen boys videos

    sex and the city

    ‘pitts assets

    Japanese massage videos

    “twerkers paradise

    3x: return of xander cage full hd movie
    jack 1994 :
    hayley anderson nester photos

    Jonathan fields

    les misérables showtimes san francisco

    michael h

    fifty shades darker
    eee

    1p) js! - mansoura news

    Label Decision
    ‘1 Sensitive
    2- Acceptable
    2- Acceptable
    2- Acceptable
    2- Acceptable z

    2- Acceptable
    2- Acceptable
    2 Acceptable
    2- Acceptable
    2- Acceptable
    2- Acceptable

    2- Acceptable

    2- Acceptable

    2- Acceptable

    Sensitive Category (if
    applicable)

    » 0- Child Exploitation

    ¥

    me u H
    sure “teen refers to 18
    Exploitation.

    The string refers to ama
    The string is not explicitly |
    The string is not explicitly sexual in nate

    Aside search shows that there is.
    this is not considered a Sensitive

    Though “xxx” often refers to topics ¢
    shows that this string refers to a film and

    The string contains no sensitive content.

    The string contains a name, whic
    name of 3 porn star, or 1stt

    The string contains a name,
    name of a porn star, Sr otnorween :
    The string is a proper name. Ae
    the rest of the string is in English,
    does not contain anything sensitive.

    The string is a first name with last name |
    which does not appear to be the name
    sensitive in nature,

    The string refers to a movie which has:
    this string is not considered adultco

    English. Both and

    Arabic oration: Tavares
    Arabic-speaking users often s

    worki in 118m woneows h i
    cases of transliteration such as | on

    The t spell is achuaiy
    Ine Sugeest tacks |


    Senvithes Category (it
    applicable)
    FU= Chile Exploitation

    Explanation
    Sure "teen to TH OF TF YE the correet category & ChiNd
    Exploitation
    The string refe on show,
    The string sexual in nature.
    exual in natul
    g called "twerkers paradise”, 50
    t considered a sens z
    x" often refers to topi adult nature, a side search
    that this string refers to a film and is not sensitive in nature
    tring contains no sensitive content

    ing contains a proper ¢, which does not appear to be the
    name of 2 porn star, or othe e
    The string contains a proper name, which does not appear to be the
    name of a porn star, 0 wise Sensitive in nature.
    The string is a proper name. Although “les misérables” is French and
    the rest of the string is in English, this is the title of a famous play and
    does not contain anything sensitive
    The string is a first nome with last name initial. This is a proper name
    which dots not appear to be the name of a porn star, or otherwise
    sensitive in nature.

    The string refers to a movie which has sexual undertones. However,
    this string is not considered adult content/sexual content

    Raters working in }18n workflows should not skip for strings with
    English, Both Arabic and English are considered ok to rate in the
    Arabic workflow. The string does not appear to be sensitive in nature,
    Arabic-speaking users often search using transiiterations. Raters
    working in i18n workflows should not skip for English, of for special
    cases of transliteration such as this one.

    ‘The correct spelling is actually “mont-saint-michel’, but even though:
    this suggestion string lacks the correct Myphens, itis stilt

    underst ence tie. wer meaning has not been pa ae
    also applies to names like “anne-sophie” or “jean: 3

    have hyphens, but would still be considered ponaenskne proper

    not appear sensitive in mature”

    names.
    Freenine ot tie tek weaen ratepnesr iets Albee
    in bs »

    Y)
    “”
    Lu
    a
    O
    Zz
    O
    Oo
    ae
    O
    Ww
    QO
    Lu

    yeea! I tating.
    Guidelines v2.0(08/07F2
    [CURRENT] Typeah
    Rating Guid ss
    (06/07/2029) <0




    a Integrity on-call action 6 ;

    Il Typeahead team action

    sas rattles) ves

    policy violating?



    eer eli Cd |

    es tas eeu ee Beles ied
    After clarification deemed to be violating, ee dy
    or decide we should update policy

    melt)



    Pea} TA & NLP figure out why
    isl face ce) Pel
    re)
    Losers)
    Maite i
    BYocklist/ Teles]
    SCT fail ert ta



    ty on-call take
    blocking, but no









    No better

    f Quality issue, fix in a
    tear 1)
    Daisey

    BSC ir lal ela)

    REDACTED FOR CONGRESS


    HE (Search) uploaded a file in the group: Search Integrity - Climate
    Change Escalation
    July 22, 2020. @

    Purpose:

    Creating this group to coordinate and manage all responses for the potential risk of Climate



    Change misinfo for Search. ”
    Y)
    lw

    Context: e

    There has a few public press incidents (Vox, NYTimes) this past week about FB's stance on az

    climate Change, and we've been getting direction from org leads to ensure Search is high O

    quality on this topic. - O
    ao
    O

    Planned Actions for Policy Review: =

    https://fb.quip.com/5tfAAASZ9wFZ Ww
    -E
    O

    Climate Change - Search Integrity Response a
    Quip LU
    oa
    © 6 3 Comments 1 Share Seen by 27
    a) Like @ Share
    a nh are ready to pull the trigger. Please let us know if you're ok with this
    approach.
    Like - 42w
    = ae - FYI, plan to apply blue-verified filtering on climate change videos
    Like. 42w 092

    FYI am proposing the following regexes to Wordy block (will wait for policy
    approval):

    * .(.¢limate change/climate change.).
    * .(.global warming/global warming.).
    MVE AS:
    There has a few public press incide

    Climate change, and we've been ge
    quality on this topic.

    nts (Vox, NYTimes) this past week about FB's stance on
    tting direction from org leads to ensure Search is high

    Planned Actions for Policy Review:
    https://fb.quip.com/StfAAASZQwFZ



    Climate Change - Search Integrity Response

    Quit

    hare Seen by 27

    © 6

    qd) Like f> Share

    - we are ready to pull the trigger. Please let us know if you're ok with this
    approach.

    Like

    a FYI, plan to apply blue-verified filtering on climate change videos
    Oo:

    Like 42w

    e
    FYI | am proposing the following regexes to Wordy block (will wait for policy
    approval):

    * .(.climate change/climate change.).
    * .(.global warming/global warming.).
    * .(.#globalwarming|#globalwarming.).
    * .(#climatechange/#climatechange.).
    * .(.#climatereality/#climatereality.).

    to turn off suggestions/hashtags related to global warming or climate change
    except the exact match.

    co win like to know your opinion on this!

    REDACTED FOR CONGRESS

    Like 42w Edited

    °
    s
    S
    Climate Change - Search Integrity Response
    Quip

    3 Comments 1 Share Seen by

    ©6
    qd) Like > Share
    z COMBE we are ready to pull the trigger. Please let us know if you're ok with this

    approach.
    Like - 42w By

    P| FYI, plan to apply blue-verified filtering on climate change videos

    Like 42w OD 2

    ga FYI lam Bie the following regexes to Wordy block (will wait for policy
    approval):

    * .(.climate change/climate change.).

    * .(.global warming/global warming.).

    ¢ .(.#globalwarming/#globalwarming.).
    ¢ .(#climatechange/#climatechange.).
    ¢ .(#climatereality/#climatereality.).

    to turn off suggestions/hashtags related to global warming or climate change
    except the exact match.

    cc will like to know your opinion on this!

    REDACTED FOR CONGRESS



    Like . 42w . Edited

    > Climate Science Information Center (CSIC) FYI
    February 9 - & ;

    Climate Science Search Query POC