{"id":10891,"date":"2019-04-23T17:17:45","date_gmt":"2019-04-23T20:17:45","guid":{"rendered":"https:\/\/tarciziosilva.com.br\/blog\/?p=10891"},"modified":"2019-04-29T23:37:11","modified_gmt":"2019-04-30T02:37:11","slug":"text-analysis-with-antconc-for-social-media-data-exploration-word-lists-word-frequencies-and-file-view","status":"publish","type":"post","link":"https:\/\/tarciziosilva.com.br\/blog\/text-analysis-with-antconc-for-social-media-data-exploration-word-lists-word-frequencies-and-file-view\/","title":{"rendered":"Text Analysis with AntConc for social media data: Word Lists, Word Frequencies and File View"},"content":{"rendered":"

\"\"In the last post, we learn the basics about AntConc<\/strong>. Now we are going to show you how to use AntConc to generate word lists (and frequencies) and the useful File Viewer.<\/p>\n

Don’t forget that this post is part of a series of tutorials:<\/p>\n

    \n
  1. Intro, Opening a File and Settings<\/a><\/li>\n
  2. Word Lists and File Viewer<\/strong>\u00a0(we are here)<\/li>\n
  3. Concordancer and Concordance Plot<\/a><\/li>\n
  4. Clusters and N-Grams\u00a0(soon)<\/li>\n
  5. Collocations\u00a0(soon)<\/li>\n<\/ol>\n

     <\/p>\n

    AntConc functions are accessed through the seven tabs below:<\/span><\/p>\n

    \"\"<\/p>\n

    In that basic tutorial, we are going to follow the steps to produce simple word lists.<\/span><\/p>\n

    Remember to open your file and import the recommended settings for social media research [<\/span>tutorial<\/span><\/a>].<\/span><\/p>\n

    Generating and navigating in a simple Word List<\/i><\/strong><\/p>\n

      \n
    1. Open your file(s). In the examples below I\u2019m going to use a dataset with 16k tweets in english containing the word \u2018brazil\u2019 (collected through Netlytic). Download the file <\/span>brazil_tweets_16732tweets_2017_11_30.txt<\/span><\/i> in <\/span>our folder<\/span><\/a>.<\/span><\/li>\n<\/ol>\n

       <\/p>\n

      2. In the <\/span>Word List<\/b> tab, click on <\/span>Start<\/span><\/i> and wait a few seconds\u00a0<\/span><\/p>\n

      \"\"<\/a><\/p>\n

      \"\"<\/a><\/p>\n

      3. Now you can explore and navigate the data, scrolling down to find meaningful words, sort by Frequency, Word (alphabetical order) or Word end.<\/span><\/p>\n

      \"\"<\/a><\/p>\n

      4. You can search for a term on the box at the bottom and click on the button \u201cSearch Only\u201d:<\/span><\/p>\n

      \"\"<\/a><\/p>\n

       <\/p>\n

      5. If you click on any word, you\u2019ll be directed to the Concordance tab. You can also read <\/span>a tutorial on the Concordance tool (soon)<\/span><\/p>\n

      \"\"<\/a><\/p>\n

       <\/p>\n

      6. And if you click in any word on the Concordance tool, you\u2019ll be directed to the <\/span>File View<\/span><\/i> tool. It functions like a simple text reader, where you can see the full corpora.<\/span><\/p>\n

      \"\"<\/a><\/p>\n

       <\/p>\n

      7. <\/span>To export the list, just go back to the <\/span>Word List<\/span><\/i> tab and click on <\/span>File -> Save Output<\/span><\/i>.<\/span><\/p>\n

      \"\"<\/a><\/p>\n

       <\/p>\n

      8.\u00a0<\/span>The output is a .txt file that looks like this:<\/span><\/p>\n

      \"\"<\/a><\/p>\n

      9. Then you can open or copy-paste the output in a spreadsheet software like Excel or Libreoffice to further analyses.<\/span><\/p>\n

      \"\"<\/p>\n

       <\/p>\n

       <\/p>\n

      Filtering out stopwords<\/i><\/strong><\/p>\n

      Stopwords are words that you <\/span>don\u2019t<\/b> want to count or visualize. Usually, they are the most common words without semantic or topical relevance for your research problem (such as articles, pronouns and some adverbs).<\/span><\/p>\n

       <\/p>\n

        \n
      1. First, you need a stopword list! You can produce or edit a list yourself, but let\u2019s start with an example list. You can download it on the <\/span>lists folder<\/span><\/a>.<\/span><\/li>\n<\/ol>\n

        \"\"<\/a><\/p>\n

         <\/p>\n

        2. To upload a stopwords lists from a .txt file, go to <\/span>Tool Preferences -> Word List<\/span><\/i>. There you\u2019ll see the option \u201cUse a stoplist below\u201d in the \u201cWord List Range\u201d section. Click on <\/span>Open<\/span><\/i> and select your .txt file.<\/span><\/p>\n

        \"\"<\/a><\/p>\n

         <\/p>\n

        If you have done it right, the words will show in the box:<\/span><\/p>\n

        \"\"<\/a><\/p>\n

         <\/p>\n

        Now you just click on \u201cApply\u201d!<\/span><\/p>\n

          \n
        1. Go back to the <\/span>Word List<\/span><\/i> tab and click again the button \u201cStart\u201d. Compare the two word lists below. The first one was the original word list and the second one is the list with stopwords filtered out:<\/span><\/li>\n<\/ol>\n

          \"\"<\/a><\/p>\n

          \"\"<\/a><\/p>\n

           <\/p>\n

           <\/p>\n

          Counting specific words<\/i><\/strong><\/p>\n

          Other useful Word List option is to count only specific words that you already know or that you just discovered in your corpora\/datasets. Follow the steps below:<\/span><\/p>\n

          1. Firstly, you\u2019ll need a word list. In our case, we are going to upload a list of words of brazilian soccer teams like that below:<\/span><\/p>\n

          \"\"<\/a><\/p>\n

           <\/p>\n

          2. Go to <\/span>Tool Preferences<\/span><\/i> -> <\/span>Word Lists<\/span><\/i> and open the Words from the file (download it in the <\/span>lists folder<\/span><\/a>). Click on \u201c<\/span>Use specific words below<\/span><\/i>\u201d and Apply.<\/span><\/p>\n

          \"\"<\/a><\/p>\n

           <\/p>\n

          3.\u00a0Go to the Word List<\/i> tab and click on \u2018Start\u2019 to generate the list again. The result will be a list of only the desired words:<\/p>\n

          \"\"<\/a><\/p>\n

           <\/p>\n

          4. Exporting that list (through <\/span>File<\/span><\/i> -> <\/span>Save Output<\/span><\/i>) and you can produce a Treemap like that visualization below with <\/span>RAW Graphs<\/span><\/a>:<\/span><\/p>\n

          \"\"<\/a><\/p>\n

           <\/p>\n

           <\/p>\n

          Counting Lemma Word Forms<\/i><\/strong><\/p>\n

          This is a <\/span>optional<\/span><\/i> step, if you want to aggregate the inflected forms of a word. For example, the verb <\/span>talk<\/span><\/i> may appear as <\/span>talking<\/span><\/i>, <\/span>talks<\/span><\/i>, <\/span>talking<\/span><\/i> and so on. Lemmatization aggregates those inflected forms to <\/span>talk<\/span><\/i>.<\/span><\/p>\n

          On social media data, this is important to investigate variations of a same root meaning, such as <\/span>autism<\/span><\/i>, <\/span>autistic<\/span><\/i>, \u201c<\/span>autist\u201d<\/span><\/i> related to a search query for <\/span>vaccines<\/span><\/i> for example.<\/span><\/p>\n

           <\/p>\n

          Steps:<\/span><\/p>\n

            \n
          1. In AntConc, the first thing you\u2019ll need is a lemma list. You can download it directly from the <\/span>AntConc website<\/span><\/a> or in our <\/span>wordlists folder<\/span><\/a>. The file will look like this:<\/span><\/li>\n<\/ol>\n

            \"\"<\/a><\/p>\n

             <\/p>\n

            That means you can add or remove lines of lemma correspondences. As you can see above, that list doesn\u2019t include the word autism. We could add the following line:<\/span><\/p>\n

            autism<\/em> -> autistic<\/em>, autistically<\/em>, autist<\/em><\/span><\/p>\n

            (even though the word \u2018autist\u2019 doesn\u2019t exists, it could be added because it is a common error between portuguese speakers, for example)<\/span><\/p>\n

             <\/p>\n

            2. To add a <\/span>Lemma List<\/span><\/i> you just need go to <\/span>Tool Preferences<\/span><\/i> -> <\/span>Word List<\/span><\/i> and click <\/span>Load<\/span><\/i> on Lemma List options.<\/span><\/p>\n

            \"\"<\/a><\/p>\n

             <\/p>\n

            After you selected your file, AntConc will show you a preview. Click in \u201cOK\u201d and then \u201cApply\u201d:<\/span><\/p>\n

            \"\"<\/a><\/p>\n

             <\/p>\n

            3. Now you can go back to the <\/span>Word List<\/span><\/i> tab and generate your list again. As you can see below, now AntConc counts <\/span>Lemma Types<\/span><\/i> and <\/span>Lemma Tokens<\/span><\/i> instead of <\/span>Word Types<\/span><\/i> and <\/span>Word Tokens<\/span><\/i>:<\/span><\/p>\n

            \"\"<\/a><\/p>\n

            Lemmatization can greatly improve the preciseness of some claims about your corpora.<\/p>\n

            We hope Word lists, Word Frequencies, Filtering Stop Words and Lemmatization techniques will help you to explore and analyze your social media datasets.<\/p>\n

             <\/p>\n

            The next AntConc tutorial will focus on\u00a0Concordancer and Concordance Plot (soon)!<\/p>\n","protected":false},"excerpt":{"rendered":"

            In the last post, we learn the basics about AntConc. Now we are going to show you how to use AntConc to generate word lists (and frequencies) and the useful File Viewer. Don’t forget that this post is part of a series of tutorials: Intro, Opening a File and Settings Word Lists and File Viewer\u00a0(we […]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,108],"tags":[2370,2584,2583,2582,2586,2580,2585,2579],"class_list":["post-10891","post","type-post","status-publish","format-standard","hentry","category-midias-sociais","category-pesquisa","tag-corpus-linguistics","tag-filtering-stop-words","tag-lemmas","tag-lemmatization","tag-social-media-corpora","tag-social-media-data","tag-stoplist","tag-text-analysis"],"_links":{"self":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts\/10891","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/comments?post=10891"}],"version-history":[{"count":12,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts\/10891\/revisions"}],"predecessor-version":[{"id":10962,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts\/10891\/revisions\/10962"}],"wp:attachment":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/media?parent=10891"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/categories?post=10891"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/tags?post=10891"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}