{"id":10965,"date":"2019-05-02T21:15:59","date_gmt":"2019-05-03T00:15:59","guid":{"rendered":"https:\/\/tarciziosilva.com.br\/blog\/?p=10965"},"modified":"2019-05-02T21:38:50","modified_gmt":"2019-05-03T00:38:50","slug":"text-analysis-with-antconc-for-social-media-data-clusters-and-n-grams","status":"publish","type":"post","link":"https:\/\/tarciziosilva.com.br\/blog\/text-analysis-with-antconc-for-social-media-data-clusters-and-n-grams\/","title":{"rendered":"Text Analysis with AntConc for social media data: Clusters and N-Grams"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignright wp-image-10864\" src=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/04\/antconc-297x300.png\" alt=\"\" width=\"180\" height=\"182\" srcset=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/04\/antconc-297x300.png 297w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/04\/antconc.png 499w\" sizes=\"auto, (max-width: 180px) 100vw, 180px\" \/>In the last post, we learn how to explore and analyze concordances in the software\u00a0<strong>AntConc<\/strong>. Now we are going to show you how to use AntConc to study clusters, n-grams and also how to count hashtags frequency and locate them.<\/p>\n<p>This post is part of a series of tutorials:<\/p>\n<ol>\n<li><a href=\"https:\/\/tarciziosilva.com.br\/blog\/text-analysis-with-antconc-for-social-media-data-exploration-intro-file-and-settings\" target=\"_blank\" rel=\"noopener\">Intro, Opening a File and Settings<\/a><\/li>\n<li><a href=\"https:\/\/tarciziosilva.com.br\/blog\/text-analysis-with-antconc-for-social-media-data-exploration-word-lists-word-frequencies-and-file-view\/\" target=\"_blank\" rel=\"noopener\">Word Lists and File Viewer\u00a0<\/a><\/li>\n<li><a href=\"https:\/\/tarciziosilva.com.br\/blog\/text-analysis-with-antconc-for-social-media-data-concordancer-and-concordance-plot\/\" target=\"_blank\" rel=\"noopener\">Concordancer and Concordance Plot\u00a0<\/a><\/li>\n<li>Clusters and N-Grams (we are here)<\/li>\n<li>Collocations\u00a0(soon)<\/li>\n<li>Keyword Lists\u00a0(soon)<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<p><strong><i>Extracting N-Grams from the corpora<\/i><\/strong><\/p>\n<p><span style=\"font-weight: 400;\">N-Grams are a contiguous sequence of items from a text(s). They can be used as a way to analyze phonemes, letters, syllables or words, for example. To our goals, we are going to see the required steps to generate n-grams of words in a given text.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">1. Firstly, as usual, you need to open a file and generate a Word List. This time, we are going to use a corpus of \u00a0thousand tweets containing the term \u2018<\/span><i><span style=\"font-weight: 400;\">plastic<\/span><\/i><span style=\"font-weight: 400;\">\u2019. Download a <\/span><a href=\"https:\/\/drive.google.com\/drive\/u\/0\/folders\/1GXHjhuWFDOkLICq7mJ5Tis3cJmu0WXzJ\"><span style=\"font-weight: 400;\">corpus file with 19k tweets<\/span><\/a>.<\/p>\n<p><span style=\"font-weight: 400;\">2.\u00a0<\/span><span style=\"font-weight: 400;\">It\u2019s important that you don\u2019t forget to open the <\/span><a href=\"https:\/\/tarciziosilva.com.br\/blog\/text-analysis-with-antconc-for-social-media-data-exploration-intro-file-and-settings\"><span style=\"font-weight: 400;\">customized settings<\/span><\/a><span style=\"font-weight: 400;\">, because we\u2019ll work with #hashtags and @usernames. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">3. T<\/span><span style=\"font-weight: 400;\">o use <\/span><i><span style=\"font-weight: 400;\">N-Grams<\/span><\/i><span style=\"font-weight: 400;\">, you need to go to the tab <\/span><i><span style=\"font-weight: 400;\">Clusters\/N-Grams<\/span><\/i><span style=\"font-weight: 400;\"> and check the option highlighted on the image:<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-10966 size-full\" src=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram.png\" alt=\"\" width=\"512\" height=\"387\" srcset=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram.png 512w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-150x113.png 150w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-300x227.png 300w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">4. Click and Start and here it is the result<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-10968 size-full\" src=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-results.png\" alt=\"\" width=\"512\" height=\"386\" srcset=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-results.png 512w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-results-150x113.png 150w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-results-300x226.png 300w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">5. You can change the minimum and maximum values of the N-Gram Size. For example, if you change them to 1 and 3, the result changes and now you can see 1-gram and 3-gram sequences of words:<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-10967 size-full\" src=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-1-gram-and-3-gram.png\" alt=\"\" width=\"512\" height=\"390\" srcset=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-1-gram-and-3-gram.png 512w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-1-gram-and-3-gram-150x114.png 150w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-1-gram-and-3-gram-300x229.png 300w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">6. To analyze specific clusters of words, you can search for them using the Search Term box.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-10969 size-full\" src=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antcon-n-gram-search.png\" alt=\"\" width=\"512\" height=\"391\" srcset=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antcon-n-gram-search.png 512w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antcon-n-gram-search-150x115.png 150w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antcon-n-gram-search-300x229.png 300w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>7.\u00a0<span style=\"font-weight: 400;\">Mind the options like <\/span><i><span style=\"font-weight: 400;\">Search Term Position<\/span><\/i><span style=\"font-weight: 400;\">. You can search for words before or after your terms. Searching for clusters with size 2 with the term <\/span><i><span style=\"font-weight: 400;\">plastic<\/span><\/i><span style=\"font-weight: 400;\"> On Right, we get the following result.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-10970 size-full\" src=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-word-on-right.png\" alt=\"\" width=\"512\" height=\"388\" srcset=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-word-on-right.png 512w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-word-on-right-150x114.png 150w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-ngram-word-on-right-300x227.png 300w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">8. Important: the column <\/span><i><span style=\"font-weight: 400;\">Range<\/span><\/i><span style=\"font-weight: 400;\"> counts the number of Corpus Files that word or term is present. This number is specially interesting when you are comparing corpora.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><strong><i>Counting Hashtags<\/i><\/strong><\/p>\n<p><span style=\"font-weight: 400;\">You can use the <\/span><i><span style=\"font-weight: 400;\">Clusters\/N-Grams<\/span><\/i><span style=\"font-weight: 400;\"> tab to count terms that have some structure, character or word in common. For example: #hashtags and @usernames in some social media platforms.<\/span><\/p>\n<p><span>1. Open your file(s). Don\u2019t forget to configure the settings as explained in the tutorial on <\/span><a href=\"https:\/\/docs.google.com\/document\/d\/14xgXsmEjgIj_b_12B-zfbf04C0hkw238T9OIu8SxXKs\/edit\"><span>files and settings<\/span><\/a><span>.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">2. Generate a Word List.<\/span><\/p>\n<p>3.\u00a0<span style=\"font-weight: 400;\">To generate a list of hashtags, for example, you should go in <\/span><i><span style=\"font-weight: 400;\">Clusters\/N-Grams<\/span><\/i><span style=\"font-weight: 400;\"> tab and use the Search Term \u00a0\u00a0\u2018 #* \u2018 , using 1-1 values in the Cluster Size options.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-10973 size-large\" src=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-counting-hashtags-580x438.png\" alt=\"\" width=\"580\" height=\"438\" srcset=\"https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-counting-hashtags-580x438.png 580w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-counting-hashtags-150x113.png 150w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-counting-hashtags-300x227.png 300w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-counting-hashtags-768x580.png 768w, https:\/\/tarciziosilva.com.br\/blog\/wp-content\/uploads\/2019\/05\/antconc-counting-hashtags.png 984w\" sizes=\"auto, (max-width: 580px) 100vw, 580px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">Troubleshooting: counting hashtags didn\u2019t worked? Probably, AntConc is configured with the default settings, where the \u2018#\u2019 symbol is a wildcard. Check! <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the last post, we learn how to explore and analyze concordances in the software\u00a0AntConc. Now we are going to show you how to use AntConc to study clusters, n-grams and also how to count hashtags frequency and locate them. This post is part of a series of tutorials: Intro, Opening a File and Settings [&hellip;]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,108],"tags":[2581,2589,2588,2586,2579],"class_list":["post-10965","post","type-post","status-publish","format-standard","hentry","category-midias-sociais","category-pesquisa","tag-antconc","tag-clusters","tag-n-gram","tag-social-media-corpora","tag-text-analysis"],"_links":{"self":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts\/10965","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/comments?post=10965"}],"version-history":[{"count":4,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts\/10965\/revisions"}],"predecessor-version":[{"id":10983,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/posts\/10965\/revisions\/10983"}],"wp:attachment":[{"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/media?parent=10965"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/categories?post=10965"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tarciziosilva.com.br\/blog\/wp-json\/wp\/v2\/tags?post=10965"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}