2.3 Python Tokens (BT101CO)

In a Python program, Tokens are the smallest individual units that the interpreter recognizes. Just as words and punctuation marks form sentences in English, tokens are the building blocks that form Python statements.

1. Keywords

Keywords are reserved words that have a fixed, predefined meaning for the Python interpreter. They cannot be used as names for variables, functions, or any other identifiers.

  • Examples: if, else, while, for, break, import, def, return, True, False.
  • Note: Python keywords are case-sensitive.

2. Identifiers

Identifiers are names given to various program elements, such as variables, functions, classes, and objects. The book outlines specific rules for naming identifiers:

  • Must begin with a letter (A-Z, a-z) or an underscore (_).
  • Can be followed by any number of letters, digits (0-9), or underscores.
  • Cannot be a keyword.
  • Special symbols (except underscore) like @, $, and % are not allowed.
  • Examples: student_name, total_sum, _temp.

3. Literals (Values)

Literals represent fixed data values in a program. The Kamthanes categorize them by the type of data they hold:

  • String Literals: Text enclosed in quotes (e.g., "Ashok", 'Python').
  • Numeric Literals: Whole numbers or decimals (e.g., 100, 3.14).
  • Boolean Literals: True and False.
  • Special Literal: None, used to indicate the absence of a value.

4. Operators

Operators are special symbols used to perform computations on data (operands). The textbook focuses on several categories:

  • Arithmetic Operators: +, -, *, /, % (modulus), ** (exponent), // (floor division).
  • Relational Operators: ==, !=, <, >, <=, >=.
  • Logical Operators: and, or, not.
  • Assignment Operators: =, +=, -=, etc.

5. Punctuators (Delimiters)

Punctuators are symbols used to organize the code structure and indicate the boundaries of expressions or blocks.

  • Common Punctuators: ( ), [ ], { }, @, ,, :, ., =, ;.
  • For example, the colon (:) is used at the end of if or for headers to signify the start of an indented block.

Comparison of Tokens in a Statement

To illustrate how these tokens work together, consider this line of code:

if amount > 100 :

Token Type
if Keyword
amount Identifier
> Operator (Relational)
100 Literal (Numeric)
: Punctuator

By recognizing these tokens, the Python interpreter understands that it needs to check if the value stored in the variable amount is greater than the constant 100.

Practice Quiz