import nltk
from nltk.corpus import wordnet

# Ensure you have the WordNet data
nltk.download('wordnet')

def find_associations(word):
    synsets = wn.synsets(word)
    associations = {
        'synonyms': set(),
        'hypernyms': set(),
        'hyponyms': set(),
        'similar_words': set()
    }

    for synset in synsets:
        # Get the formatted synset name (e.g., 'trout.n.02')
        synset_name = synset.name()

        # Get synonyms
        synonyms = synset.lemma_names()
        for synonym in synonyms:
            associations['synonyms'].add(f"{synonym} ({synset_name})")

        # Get hypernyms
        for hypernym in synset.hypernyms():
            hypernym_name = hypernym.name()
            associations['hypernyms'].add(f"{hypernym_name} ({hypernym_name})")

        # Get hyponyms
        for hyponym in synset.hyponyms():
            hyponym_name = hyponym.name()
            associations['hyponyms'].add(f"{hyponym_name} ({hyponym_name})")

        # Get similar words
        for similar in synset.similar_tos():
            similar_name = similar.name()
            associations['similar_words'].add(f"{similar_name} ({similar_name})")

    return associations

# Find associations for "positive"
positive_associations = find_associations('positive')
print("Associations for 'positive':")
for relation_type, words in positive_associations.items():
    print(f"{relation_type.capitalize()}: {list(words)}")

# Find associations for "trust"
trust_associations = find_associations('confident')
print("\nAssociations for 'confident':")
for relation_type, words in trust_associations.items():
    print(f"{relation_type.capitalize()}: {list(words)}")

output:

Associations for 'positive':
Synonyms: ['positive (positive.a.08)', 'irrefutable (incontrovertible.s.01)', 'prescribed (positive.s.05)', 'positively_charged (positive.s.10)', 'electropositive (positive.s.10)', 'confirming (positive.a.04)', 'positive (cocksure.s.01)', 'positive (positive.s.05)', 'positive (positive.n.01)', 'overconfident (cocksure.s.01)', 'plus (plus.s.02)', 'confident (convinced.s.01)', 'positive (positive.s.10)', 'positivistic (positivist.a.01)', 'positivist (positivist.a.01)', 'positive (incontrovertible.s.01)', 'cocksure (cocksure.s.01)', 'positive (positivist.a.01)', 'positive_degree (positive.n.01)', 'positive (plus.s.02)', 'positive (positive.a.01)', 'positive (convinced.s.01)', 'positive (positive.a.04)', 'convinced (convinced.s.01)', 'positive (positive.n.02)', 'incontrovertible (incontrovertible.s.01)', 'positive (positive.s.09)']
Hypernyms: ['film.n.03 (film.n.03)', 'adverb.n.02 (adverb.n.02)', 'adjective.n.01 (adjective.n.01)']
Hyponyms: []
Similar_words: ['advantageous.a.01 (advantageous.a.01)', 'formal.a.01 (formal.a.01)', 'certain.a.02 (certain.a.02)', 'affirmative.s.02 (affirmative.s.02)', 'plus.a.01 (plus.a.01)', 'gram-positive.s.01 (gram-positive.s.01)', 'constructive.s.02 (constructive.s.02)', 'confident.a.01 (confident.a.01)', 'charged.a.01 (charged.a.01)', 'undeniable.a.01 (undeniable.a.01)']

Associations for 'confident':
Synonyms: ['surefooted (confident.s.03)', 'confident (confident.a.01)', 'positive (convinced.s.01)', 'convinced (convinced.s.01)', 'confident (convinced.s.01)', 'confident (confident.s.03)', 'sure-footed (confident.s.03)']
Hypernyms: []
Hyponyms: []
Similar_words: ['reassured.s.01 (reassured.s.01)', 'certain.a.02 (certain.a.02)', 'capable.a.01 (capable.a.01)', 'self-assured.s.01 (self-assured.s.01)', 'cocksure.s.01 (cocksure.s.01)', 'assured.s.01 (assured.s.01)']

Breakdown of the Output

e.g. Synonyms: ['positive (positive.a.08)', 'irrefutable (incontrovertible.s.01)', 'prescribed (positive.s.05)', 'positively_charged (positive.s.10)', 'electropositive (positive.s.10)', 'confirming (positive.a.04)', 'positive (cocksure.s.01)', 'positive (positive.s.05)', 'positive (positive.n.01)', 'overconfident (cocksure.s.01)', 'plus (plus.s.02)', 'confident (convinced.s.01)', 'positive (positive.s.10)', 'positivistic (positivist.a.01)', 'positivist (positivist.a.01)', 'positive (incontrovertible.s.01)', 'cocksure (cocksure.s.01)', 'positive (positivist.a.01)', 'positive_degree (positive.n.01)', 'positive (plus.s.02)', 'positive (positive.a.01)', 'positive (convinced.s.01)', 'positive (positive.a.04)', 'convinced (convinced.s.01)', 'positive (positive.n.02)', 'incontrovertible (incontrovertible.s.01)', 'positive (positive.s.09)']

1. General Format:

   • Each entry in the list has the format word (synset_name), where:

     • word is the synonym.

     • synset_name indicates the synset to which the word belongs, with:

       • The first part (e.g., positive) being the lemma (the base form of the word).

       • The second part (e.g., a for adjective, n for noun) indicating the part of speech.

       • The last part (e.g., 01, 02) indicating the sense number, which distinguishes between different meanings of the same word.

2. Example Entries:

   • 'positive (positive.a.08)':

     • This means that "positive" is an adjective (indicated by a) and belongs to the eighth sense of the word "positive" in WordNet.

   • 'irrefutable (incontrovertible.s.01)':

     • "Irrefutable" is a synonym for "positive" and is associated with the first sense of the adjective "incontrovertible" (s indicating it's an adjective).

   • 'prescribed (positive.s.05)':

     • "Prescribed" is a synonym that is associated with the fifth sense of "positive."

   • 'confident (convinced.s.01)':

     • "Confident" is synonymous with "positive" and is linked to the first sense of "convinced."

3. Multiple Entries:

   • Some synonyms appear multiple times with different senses. For instance, "positive" itself appears several times with different sense numbers (positive.a.01, positive.s.05, etc.), indicating that it has multiple meanings or usages in different contexts.

4. Diverse Synonyms:

   • The list includes a mix of direct synonyms (words that can often replace "positive" in a sentence) and related terms that capture similar meanings or connotations but may not be interchangeable in all contexts.

[2] Find word association, return associated synsets

def find_associations(word):
    synsets = wn.synsets(word)
    associations = {
        'synonyms': set(),
        'hypernyms': set(),
        'hyponyms': set(),
        'similar_words': set()
    }

    for synset in synsets:
        # Get synonyms
        synonyms = synset.lemma_names()
        associations['synonyms'].update(synonyms)

        # Get hypernyms
        for hypernym in synset.hypernyms():
            associations['hypernyms'].update(hypernym.lemma_names())

        # Get hyponyms
        for hyponym in synset.hyponyms():
            associations['hyponyms'].update(hyponym.lemma_names())

        # Get similar words
        for similar in synset.similar_tos():
            associations['similar_words'].update(similar.lemma_names())

    return associations

# Find associations for "positive"
positive_associations = find_associations('positive')
print("Associations for 'positive':")
for relation_type, words in positive_associations.items():
    print(f"{relation_type.capitalize()}: {list(words)}")

# Find associations for "trust"
trust_associations = find_associations('confident')
print("\nAssociations for 'confident':")
for relation_type, words in trust_associations.items():
    print(f"{relation_type.capitalize()}: {list(words)}")

Output:

Associations for 'positive':
Synonyms: ['cocksure', 'irrefutable', 'prescribed', 'confident', 'positivistic', 'positive_degree', 'positivist', 'electropositive', 'incontrovertible', 'overconfident', 'plus', 'convinced', 'positive', 'positively_charged', 'confirming']
Hypernyms: ['adverb', 'photographic_film', 'adjective', 'film']
Hyponyms: []
Similar_words: ['plus', 'undeniable', 'affirmative', 'advantageous', 'sure', 'charged', 'optimistic', 'confident', 'certain', 'Gram-positive', 'constructive', 'formal']

Associations for 'confident':
Synonyms: ['sure-footed', 'confident', 'surefooted', 'convinced', 'positive']
Hypernyms: []
Hyponyms: []
Similar_words: ['cocksure', 'capable', 'assured', 'self-confident', 'sure', 'reassured', 'overconfident', 'self-assured', 'certain', 'positive']

[3] Part of Speech Codes in WordNet

1. n = Noun

   • Represents a person, place, thing, or idea.

   • Example: dog.n.01 refers to the first sense of "dog" as a noun.

2. v = Verb

   • Represents an action, occurrence, or state of being.

   • Example: run.v.01 refers to the first sense of "run" as a verb.

3. a = Adjective

   • Describes or modifies a noun, providing more information about it.

   • Example: happy.a.01 refers to the first sense of "happy" as an adjective.

4. s = Adjective Satellite

   • A type of adjective that provides additional descriptive information, often used in comparative forms.

   • Example: taller.s.01 refers to the comparative form of "tall."

5. r = Adverb

   • Modifies verbs, adjectives, or other adverbs, often indicating manner, place, time, frequency, degree, etc.

   • Example: quickly.r.01 refers to the first sense of "quickly" as an adverb.

Explanation of Each Part of Speech

• Noun (n): Nouns are fundamental to language, representing entities and concepts. They can be further categorized into concrete nouns (physical objects) and abstract nouns (ideas, qualities).

• Verb (v): Verbs are crucial for expressing actions and states. They can indicate physical actions (like "run"), mental actions (like "think"), or states of being (like "exist").

• Adjective (a): Adjectives enrich language by providing descriptive details about nouns. They help to convey qualities, quantities, or characteristics, enhancing the specificity of communication.

• Adjective Satellite (s): Satellite adjectives often modify the meaning of another adjective or provide additional context. They are particularly useful when comparing or contrasting qualities.

• Adverb (r): Adverbs add depth to sentences by modifying verbs, adjectives, or other adverbs. They can describe how, when, where, or to what extent an action occurs, thereby adding nuance to the action or description.

# Make data directory if it doesn't exist
!mkdir -p data
!wget -nc https://nyc3.digitaloceanspaces.com/ml-files-distro/v1/upshot-trump-emolex/data/NRC-emotion-lexicon-wordlevel-alphabetized-v0.92.txt -P data

Define processing task:

import nltk
from nltk.sentiment import SentimentIntensityAnalyzer

#nltk.download('vader_lexicon')

def load_emolex_lexicon():
    emolex_lexicon = {}
    with open('/content/data/NRC-emotion-lexicon-wordlevel-alphabetized-v0.92.txt', 'r') as file:
        for line in file:
            line = line.strip()
            if line:
                try:
                    word, emotion, value = line.split('\t')
                    if emotion == 'positive' and float(value) > 0.0:
                        emolex_lexicon[word] = 1
                    elif emotion == 'negative' and float(value) > 0.0:
                        emolex_lexicon[word] = -1
                except ValueError:
                    pass
    return emolex_lexicon

def analyze_sentiment(text):
    sia = SentimentIntensityAnalyzer()
    sentiment_scores = sia.polarity_scores(text)
    emolex_lexicon = load_emolex_lexicon()
    emolex_scores = {
        'pos': sum(sentiment_scores[word] for word in sentiment_scores if word in emolex_lexicon and sentiment_scores[word] > 0),
        'neg': sum(sentiment_scores[word] for word in sentiment_scores if word in emolex_lexicon and sentiment_scores[word] < 0),
        'neu': sum(sentiment_scores[word] for word in sentiment_scores if word not in emolex_lexicon),
        'compound': sentiment_scores['compound']
    }
    return emolex_scores

text = "This is a great day!"

scores = analyze_sentiment(text)
print(scores)

import nltk
nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')
nltk.download('wordnet')

Analyze text:

import nltk
from nltk.corpus import sentiwordnet as swn
from nltk.sentiment import SentimentIntensityAnalyzer

nltk.download('sentiwordnet')

def map_sentiment(word, pos):
    try:
        senti_synsets = list(swn.senti_synsets(word, pos))
        if len(senti_synsets) > 0:
            sentiment = sum(
                [senti_synset.pos_score() - senti_synset.neg_score() for senti_synset in senti_synsets]
            ) / len(senti_synsets)
            if sentiment > 0:
                return 'pos'
            elif sentiment < 0:
                return 'neg'
    except KeyError:
        pass
    return 'neu'

def analyze_sentiment(text):
    sia = SentimentIntensityAnalyzer()
    sentiment_scores = sia.polarity_scores(text)
    tokenized_text = nltk.word_tokenize(text)
    tagged_text = nltk.pos_tag(tokenized_text)
    sentiment_scores_swn = [
        map_sentiment(token, pos) for token, pos in tagged_text
    ]
    sentiment_scores_combined = {
        'pos': sentiment_scores['pos'] + sentiment_scores_swn.count('pos'),
        'neg': sentiment_scores['neg'] + sentiment_scores_swn.count('neg'),
        'neu': sentiment_scores['neu'] + sentiment_scores_swn.count('neu'),
        'compound': sentiment_scores['compound']
    }
    return sentiment_scores_combined

text = "This is a great day!"
scores = analyze_sentiment(text)
print(scores)

Output:

{'pos': 0.594, 'neg': 0.0, 'neu': 6.406, 'compound': 0.6588}

Conclusion:

The above code provides sentiment scores for the given text "This is a great day!" using both the SentimentIntensityAnalyzer and SentiWordNet. The output sentiment scores are as follows:

• Positive sentiment score (pos): 0.594

• Negative sentiment score (neg): 0.0

• Neutral sentiment score (neu): 6.406

• Compound sentiment score (compound): 0.6588 The positive score indicates a moderately positive sentiment in the text, while the negative score is 0, suggesting no negative sentiment. The neutral score is relatively high, which means there are many words classified as neutral in the text. The compound score of 0.6588 represents an overall positive sentiment.

Please note that sentiment analysis is not always perfect and can vary depending on the context and the specific algorithms or lexicons used.

Output:

{'neg': 0.0, 'neu': 0.443, 'pos': 0.557, 'compound': 0.7424}

The sentiment scores provided for the text "I love using NLTK for natural language processing!" using the AFINN sentiment analyzer with NLTK are as follows:

• Negative sentiment score (neg): 0.0

• Neutral sentiment score (neu): 0.443

• Positive sentiment score (pos): 0.557

• Compound sentiment score (compound): 0.7424 The compound score is a normalized value that combines the positive and negative scores to give an overall sentiment score. In this case, the compound score of 0.7424 indicates a highly positive sentiment for the given text.

However, NLTK can be used in conjunction with other libraries or models to perform sentiment analysis. One common approach is to use NLTK in combination with a pre-trained sentiment analysis model, such as the VADER (Valence Aware Dictionary and sEntiment Reasoner) model.

VADER is a rule-based sentiment analysis model that is specifically designed for sentiment analysis on social media text, where conventional techniques may not perform well. NLTK provides an interface to utilize the VADER model for sentiment analysis.

Here is an example of how you can perform sentiment analysis using NLTK with the VADER model:

import nltk
from nltk.sentiment import SentimentIntensityAnalyzer

# Download the VADER lexicon (if not already downloaded)
nltk.download('vader_lexicon')

# Create an instance of the VADER sentiment analyzer
sia = SentimentIntensityAnalyzer()

# Example text
text = "I love using NLTK for natural language processing!"

# Perform sentiment analysis
sentiment = sia.polarity_scores(text)

# Print the sentiment scores
print(sentiment)

Output:

{'neg': 0.0, 'neu': 0.417, 'pos': 0.583, 'compound': 0.7901}

The polarity_scores() method of the SentimentIntensityAnalyzer class returns a dictionary with sentiment scores for the input text. The scores include positive, negative, neutral, and compound values.

The sentiment analysis results are in the form of a dictionary with four key-value pairs:

• 'neg': 0.0: This indicates the negative sentiment score for the input text. In this case, the score is 0.0, which means there is no negative sentiment detected in the text.

• 'neu': 0.417: This represents the neutral sentiment score. The score of 0.417 suggests that a significant portion of the text is considered neutral in terms of sentiment.

• 'pos': 0.583: This corresponds to the positive sentiment score. With a value of 0.583, it suggests that a considerable portion of the text is classified as positive sentiment.

• 'compound': 0.7901: The compound score is a normalized, aggregated sentiment score that combines the positive, negative, and neutral scores. The compound score ranges from -1 to 1, where values closer to 1 indicate highly positive sentiment, and values closer to -1 indicate highly negative sentiment. In this case, the compound score of 0.7901 suggests a strongly positive sentiment overall.

Based on the sentiment analysis results, the text appears to have a predominantly positive sentiment.

Please note that NLTK's sentiment analysis using VADER is a rule-based approach and may not always be suitable for all scenarios. Depending on your specific requirements, you might explore other sentiment analysis models or libraries such as TextBlob, spaCy, or custom-trained machine learning models.

The SentimentIntensityAnalyzer class in NLTK's nltk.sentiment module uses the VADER (Valence Aware Dictionary and sEntiment Reasoner) lexicon by default. However, NLTK also allows you to use other lexicons with the SentimentIntensityAnalyzer class. Here are a few lexicons that you can use:

• AFINN: The AFINN lexicon is a list of pre-computed sentiment scores for English words. Each word in the lexicon is assigned a score between -5 and +5, indicating its sentiment polarity. Positive scores indicate positive sentiment, while negative scores indicate negative sentiment.

• SentiWordNet: SentiWordNet is a lexical resource that assigns sentiment scores to synsets (sets of synonymous words) in WordNet, a large lexical database for English. It provides sentiment scores for individual words based on their senses and part-of-speech tags.

• EmoLex: The EmoLex lexicon is a collection of English words associated with eight basic emotions: anger, anticipation, disgust, fear, joy, sadness, surprise, and trust. Each word in the lexicon is labeled with one or more emotion categories.