Monday, January 30, 2012

C O N F I D E N T I A L SECTION 01 OF 03 ANKARA 004888 SIPDIS E.O. 12958: DECL: 08/22/2025 TAGS: PGOV [Internal Governmental Affairs], PREL [External Political Relations], PINS [National Security], TU [Turkey] SUBJECT: THE USE AND ABUSE OF TURKISH POLLS REF: 2003 ANKARA 004319 Classified By: POLCOUNS John Kunstadter; E.O. 12958, reasons 1.4 (b,d) ¶1. Summary. Turkish polls have had a tendency inaccurately to forecast the outcome of Turkish national elections because the majority of Turkish survey firms do not follow the basic requirements for conducting a scientific poll. Nevertheless, Turkish elites --- including leaders in the governing AKP --- pay close attention to the results and use them to shape their political calculations and behavior. End Summary. ¶2. (C) Turkish polls have had a tendency inaccurately to forecast the outcome of Turkish national elections. In 1999, for example, most of the Turkish polling firms correctly predicted that DSP would win the plurality of the vote, but only one polling firm correctly predicted that ultranationalist MHP would receive nearly one-fifth of the vote and become the second largest party in parliament. In 2002, most polls correctly predicted that AKP would win the plurality of the vote, but they failed to accurately forecast the size of AKP,s victory. ------------------------------------- HOW TO CONDUCT AN UNSCIENTIFIC SURVEY ------------------------------------- ¶3. (C) One reasons Turkish polling results are often unreliable is that political parties sometimes commission polls for propaganda purposes, not objective analysis. For example, Ankara mayor Gocek,s head advisor Murat Dogru (strictly protect) told POLOFF that prior to the November 2002 election, Gocek,s small Democratic Party (DP) commissioned a poll indicating that DP would receive around 20 percent of the vote and secure second place in the election. In reality, the party received less than 1 percent of the vote and Gocek later abandoned the DP to join AKP. ¶4. (C) The more fundamental problem, however, is that the majority of Turkish survey firms do not follow the basic requirements for conducting a scientific poll. For a poll to be scientific, every member of the population to be surveyed (e.g. all Turkish citizens over the age of 18) must have an equal chance of being included in the sample. ¶5. (C) The majority of Turkish firms, however, violate this basic tenet by conducting their surveys in only a handful of pre-selected (not randomly selected) provinces. Turkish firms also use a quota system to select interviewees at the block and household level that can seriously bias their results. Most Turkish polls, moreover, focus on urban-only samples and when the firms try to include rural residents they implement a methodology that simply replaces one form of error with another. ¶6. (C) Ibrahim Uslu of the Ankara Social Research Center (ANAR) and Ozer Sencar of Metropoll discussed their firms polling methodologies with POLOFFs. Uslu and Sencar told us that their firms follow a methodology prescribed by the Turkish State Statistical Institute (DIE). They also suggested that the vast majority of Turkish polling firms follow variations on the same basic methodology. (Comment: Their assertion is consistent with the fact that most of the firms have made similar bad predictions in recent elections. End Comment.) ¶7. (C) The methodology employed by most Turkish firms was devised by the DIE. There are 81 provinces in Turkey, but DIE has identified thirteen "representative" provinces across Turkey and recommends that Turkish polling firms conduct surveys in these thirteen regions. Sencer also told us that DIE had a list of twenty-five "representative" provinces which firms can use to conduct "more accurate" surveys. (Comment: The fact that the DIE created a second, "more accurate" set of provinces raises red flags regarding this whole unscientific procedure. End Comment.) According to Uslu and Sencar, DIE has conducted extensive research and concluded that these provinces are similar to neighboring provinces and can be used as proxies to represent the entire region. This step violates the basic requirement for a scientific survey, i.e. that all individuals in the population have an equal chance of being selected for the survey. (Comment: Moreover, even if these thirteen or twenty-five provinces were representative of their neighboring provinces at one point in time, migration and variations in economic development have altered their representativeness over time. End Comment.) ¶8. (C) Uslu and Sencar were unable or unwilling to explain to POLOFFs how their firms constructed a methodology randomly to select individuals within the thirteen "representative" provinces. Uslu and Sencar, moreover, told POLOFFs that their firms use a quota system to select individuals within households. ANAR,s interviewers are required to conduct interviews that satisfy a preset age, gender, past voting preference, and education quota. Metropoll uses an age and gender quota and then weights the data to correct for biases in past voting preferences. Uslu and Sencar also admitted that their standard polls only interview urban respondents, although they have also conducted rural polls. (Comment: We suspect they have no methodology at the sub-provincial level and their interviewers simply wander around the province,s capital city unscientifically conducting interviews until they fill their preset quota. End Comment.) ---------------------------- SCIENTIFIC POLLING IN TURKEY ---------------------------- ¶9. (C) There are a few firms, mostly based in Istanbul, that conduct scientific polls. For example, Infakto Research --- a firm co-founded in 2003 by Emre Erdogan, a well-trained PhD social scientist --- conducts scientific surveys. Erdogan told POLOFF that his firm,s polls use stratified random samples, face-to-face interviews, and cover both urban and rural areas. Erdogan showed us documentation that corroborated his assertion. ---------------------------- ANAR,S RELATIONSHIP WITH AKP ---------------------------- ¶10. (C) Uslu reconfirmed that ANAR (REFTEL) conducts surveys for the governing Justice and Development Party (AKP). State Minister Dr. Besir Atalay, an Islamist former sociology professor and university rector (dismissed from his rectorship of Kirikkale University for pursuing Islamist policies), is the founder of ANAR and Foreign Minister Gul had an office in ANAR,s building prior to the formation of the AKP. Senior AKP leaders pay close attention to ANAR,s polls. POLOFFs have repeatedly seen notebooks with ANAR,s logo on them sitting on AKP MPs, and senior officials, desks and bookshelves. On more than one occasion, moreover, AKP leaders have read from these ANAR notebooks, quoting polling data to POLOFFs in an effort to prove a certain political point. ¶11. (C) Uslu told us that ANAR initially conducted monthly polls for AKP, but after a few years of very similar results, the party has decided to cut back to bimonthly polls. Recently, AKP has asked ANAR to conduct only quarterly polls using its normal methodology, but it has also requested that ANAR supplement this with semiannual "rural" polls. The methodology behind ANAR,s rural poll, unfortunately, is just as unscientific as its normal urban-only surveys. They use ten of the same thirteen "representative regions" and they simply replace Ankara, Izmir, and Istanbul with neighboring provinces. They then apply the same methodology, but require their interviewers to fill their quotas outside the boundaries of the provincial capitals. --------------------------------- TURKISH ELTIES AND PUBLIC OPINION --------------------------------- ¶12. (C) As elites do in other countries, Turkish elites appear to pay significant attention to public opinion polls. In addition to AKP, POLOFFs have been in meetings with ANAP, DSP, DYP, MHP, and SHP leaders who refer to public opinion polls when discussing public policies and political strategies. Journalists and newspaper columnists regularly refer to public opinion polls in their articles. Moreover, Turkish civil society organizations, like the pro-EU ARI Movement and the Liberal Thinking Association (LDT), have commissioned public opinion polls. ¶13. (C) Turkish political elites do not blindly follow the results of public opinion polls. Instead, they use polling results to help them judge the relative popularity (influence) of political leader; to estimate their party's prospects for electoral success; to inform the development of their political strategy and set a policy agenda; and to gauge how well they are marketing their party and policy positions. --------------------------------------------- ---- WHAT WE CAN LEARN FROM TURKEY,S POLLING COMMUNITY --------------------------------------------- ---- ¶14. (C) Comment. AKP,s decision to cut back on the number of polls suggests that they are unlikely to call for early elections anytime soon: it would be odd to reduce polling efforts prior to an election. However, AKP,S decision to add two "rural" polls each year suggests that AKP is starting to worry about the level of its support in rural Anatolian farming communities. ¶15. (C) Comment, continued. AKP,s continued patronage of ANAR despite the obvious errors in ANAR,s methodology reveals a lack of expertise or inclination to carefully monitor how ANAR is doing its work. This is disturbing because we know that AKP,s leaders pay close attention to ANAR,s poll results. Moreover, this is part of a larger pattern plaguing AKP in a variety of technical fields besides polling. AKP,s leadership lacks the time, energy, ability, technical know-how, or inclination to engage in rigorous analysis of the advice it receives from its friends and so-called experts. ¶16. (C) Comment, continued. The poor state of the Turkish polling community is also illustrative of larger problems within Turkish society, e.g. 1) an over reliance and acceptance of state-directed solutions to complex scientific problems and 2) a desire to cut corners by replacing a tried-and-true method with a cheap, quick, and easy alternative. ¶17. (C) Comment, continued. Until more Turkish firms follow Infakto,s example and conduct scientific surveys, readers of Turkish polls should be skeptical of the results. Nonetheless, Turkish polling results can be useful if interpreted with an extra degree of caution. At a minimum, these polls are important because many Turkish politicians, academics, journalists, and other elites pay close attention to the results. End Comment. MCELDOWNEY

No comments: