docs: Further enhance DEVELOPMENT_LOG.md with detailed technical insights

2026-03-06 02:56:56 +01:00 · 2025-07-22 21:01:45 +02:00
parent 42150088c3
commit 98e49cbc2b
1 changed files with 90 additions and 12 deletions
--- a/DEVELOPMENT_LOG.md
+++ b/DEVELOPMENT_LOG.md
@@ -25,15 +25,29 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
 ## Implemented Features & Changes
 ### 1. Core Language Setup
- **Description:** Established the foundational lexical analysis, parsing, and interpretation framework for Femcode.
+- **Description:** Established the foundational lexical analysis, parsing, and interpretation framework for Femcode. This involved defining basic token types, parsing simple expressions and statements, and setting up a rudimentary execution environment.
 - **Technical Details:**
    - **`lexer.py`:** Initial implementation of `Token` class and `Lexer` with `get_next_token` for basic token recognition (integers, strings, basic operators, keywords).
    - **`parser.py`:** Defined core AST nodes (`Number`, `String`, `BinOp`, `Print`, `Assign`, `Variable`, `Block`, `IfStatement`, `WhileStatement`, `FunctionDefinition`, `FunctionCall`, `ReturnStatement`). Implemented recursive descent parsing methods (`expression`, `term`, `factor`, `parse_statement`, etc.).
    - **`interpreter.py`:** Implemented `Interpreter` class with `visit_*` methods for each AST node, defining their runtime behavior. Introduced `scope_stack` for variable management and `ReturnValue` exception for function returns.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
    - `femcode/src/interpreter.py`
 ### 2. Dictionaries and Property Access
- **Description:** Added support for creating dictionary (object) data structures and accessing their properties using dot notation.
+- **Description:** Added support for creating dictionary (object) data structures and accessing their properties using dot notation. This allows for more complex data representation.
 - **Keywords/Syntax:** `{}` (dictionary literal), `.` (property access)
 - **Technical Details:**
    - **`lexer.py`:** Added token patterns for `LBRACE` (`{`), `RBRACE` (`}`), `COLON` (`:`), and `DOT` (`.`).
    - **`parser.py`:**
        -   New AST nodes: `Dictionary` (to represent dictionary literals) and `PropertyAccess` (to represent accessing a property of an object).
        -   `parse_dictionary_literal()` method: Parses key-value pairs within `{}`. It consumes `LBRACE`, then iteratively parses `expression` (for key), `COLON`, `expression` (for value), and `COMMA` until `RBRACE` is encountered.
        -   `factor()` method: Modified to recognize `LBRACE` for dictionary literals and `DOT` for property access. It now correctly dispatches to `parse_dictionary_literal()` and `parse_property_access()`. 
        -   `parse_property_access(target_node)` method: Consumes `DOT`, then an `ID` token for the property name, creating a `PropertyAccess` AST node.
    - **`interpreter.py`:**
        -   `visit_Dictionary(node)`: Evaluates key and value expressions for each pair and constructs a Python dictionary.
        -   `visit_PropertyAccess(node)`: Evaluates the `target` node (expected to be a dictionary) and retrieves the property using `target.get(property_name)`. Includes type checking to ensure the target is a dictionary.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
@@ -41,21 +55,35 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
    - `femcode/examples/dictionaries.fem`
 ### 3. Input/Output (`ask` built-in function)
- **Description:** Introduced a built-in function `ask` to prompt the user for input, enhancing interactivity.
+- **Description:** Introduced a built-in function `ask` to prompt the user for input, enhancing interactivity. This allows Femcode programs to receive data from the user.
 - **Keywords/Syntax:** `ask(prompt_string)`
 - **Technical Details:**
    - **`interpreter.py`:**
        -   Added `_ask_builtin(prompt)` method: This method directly uses Python's `input()` function to get user input.
        -   Modified `Interpreter.__init__`: Registered `_ask_builtin` as a callable function under the name "ask" in the `self.functions` dictionary.
        -   Modified `visit_FunctionCall(node)`: Added logic to check if `func_info` is a callable (i.e., a built-in function). If so, it calls the function directly with evaluated arguments. Otherwise, it proceeds with the logic for user-defined functions.
 - **Files Modified:**
    - `femcode/src/interpreter.py`
    - `femcode/examples/io.fem`
 ### 4. Built-in Functions (`len`, `type`)
- **Description:** Added utility built-in functions `len` for getting the length of strings/lists and `type` for determining the type of a value.
+- **Description:** Added utility built-in functions `len` for getting the length of strings/lists and `type` for determining the type of a value. These functions provide basic introspection and data manipulation capabilities.
 - **Keywords/Syntax:** `len(value)`, `type(value)`
 - **Technical Details:**
    - **`interpreter.py`:**
        -   Added `_len_builtin(obj)` method: Uses Python's built-in `len()` function. Marked as `@staticmethod`.
        -   Added `_type_builtin(obj)` method: Uses Python's `type(obj).__name__` to get the type string. Marked as `@staticmethod`.
        -   Modified `Interpreter.__init__`: Registered `_len_builtin` and `_type_builtin` in the `self.functions` dictionary.
 - **Files Modified:**
    - `femcode/src/interpreter.py`
    - `femcode/examples/built_ins.fem`
 ### 5. `femterpreter` Wrapper Script
- **Description:** Created a standalone executable wrapper script to run Femcode programs directly from the command line, simplifying execution and removing the need to specify `python3 main.py`.
+- **Description:** Created a standalone executable wrapper script to run Femcode programs directly from the command line, simplifying execution and removing the need to specify `python3 main.py`. This greatly improves the user experience.
 - **Technical Details:**
    - **`femcode/femterpreter` (new file):** A bash script that calls the compiled Python executable (`femcode/dist/femcode_linux`) with the provided arguments. It handles path resolution to ensure the executable is found.
    - **`femcode/src/main.py`:** Modified to accept the Femcode file path as a command-line argument using `sys.argv`. Includes basic error handling for missing files.
    - **Build Process:** Instructions were provided for creating a virtual environment, installing `pyinstaller`, and compiling the Linux executable (`femcode/dist/femcode_linux`).
 - **Files Modified:**
    - `femcode/femterpreter` (new file)
    - `femcode/src/main.py`
@@ -63,13 +91,23 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
 ### 6. Enhanced Data Types
 #### Floating-Point Numbers
 - **Description:** Extended numerical support to include decimal values, allowing for more precise calculations.
 - **Technical Details:**
    - **`lexer.py`:** Modified the `isdigit()` block to check for a decimal point (`.`) and subsequent digits, tokenizing them as `FLOAT`. The `token_patterns` list was updated to include `FLOAT` regex.
    - **`parser.py`:** The `factor()` method was updated to recognize `FLOAT` tokens and create `Number` AST nodes (reusing the existing `Number` AST node, as the interpreter can handle both `int` and `float` Python types).
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
 #### Null/None Type
- **Description:** Introduced a specific keyword to represent the absence of a value.
+- **Description:** Introduced a specific keyword (`Ghosted`) to represent the absence of a value, similar to `null` or `None` in other languages.
 - **Keywords/Syntax:** `Ghosted`
 - **Technical Details:**
    - **`lexer.py`:** Added a regex pattern to recognize the `Ghosted` keyword and tokenize it as `NULL`. This pattern was added to the `token_patterns` list.
    - **`parser.py`:**
        -   New AST node: `Null` (a simple placeholder AST node).
        -   `factor()` method: Modified to recognize `NULL` tokens and create `Null` AST nodes.
    - **`interpreter.py`:**
        -   `visit_Null(node)`: Returns Python's `None` when a `Null` AST node is visited.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
@@ -78,6 +116,14 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
 #### `for` Loops
 - **Description:** Implemented iteration over sequences (lists and strings), enabling more flexible looping constructs.
 - **Keywords/Syntax:** `Tomgirl` (for loop keyword)
 - **Technical Details:**
    - **`lexer.py`:** Added a regex pattern to recognize `Tomgirl` and tokenize it as `FOR`. This pattern was added to the `token_patterns` list.
    - **`parser.py`:**
        -   New AST node: `ForStatement` (stores the loop variable name, the iterable expression, and the loop body).
        -   `parse_statement()`: Modified to recognize `FOR` and dispatch to `parse_for_statement()`. 
        -   `parse_for_statement()`: Parses the loop variable (ID), the `ASSIGN` token (acting as 'in' for now, a more specific 'IN' token could be introduced later), the iterable expression, and the loop body (block).
    - **`interpreter.py`:**
        -   `visit_ForStatement(node)`: Evaluates the `iterable`. It then iterates through each item, creates a new scope for each iteration, assigns the item to the loop variable in that scope, and then visits the loop body. Includes type checking to ensure the iterable is a list or string.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
@@ -85,8 +131,18 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
    - `femcode/examples/loops_advanced.fem`
 #### `break` and `continue` Statements
- **Description:** Added statements to provide more granular control over loop execution, allowing early exit or skipping iterations.
+- **Description:** Added statements to provide more granular control over loop execution, allowing early exit (`Break`) or skipping to the next iteration (`Continue`).
 - **Keywords/Syntax:** `Break`, `Continue`
 - **Technical Details:**
    - **`lexer.py`:** Added regex patterns to recognize `Break` and `Continue` keywords, tokenizing them as `BREAK` and `CONTINUE` respectively. These patterns were added to the `token_patterns` list.
    - **`parser.py`:**
        -   New AST nodes: `BreakStatement` and `ContinueStatement` (simple marker nodes).
        -   `parse_statement()`: Modified to recognize `BREAK` and `CONTINUE` tokens and create the corresponding AST nodes.
    - **`interpreter.py`:**
        -   New exception classes: `BreakLoop(Exception)` and `ContinueLoop(Exception)`. These are custom exceptions raised by `visit_BreakStatement` and `visit_ContinueStatement` respectively.
        -   `visit_BreakStatement(node)`: Raises `BreakLoop`.
        -   `visit_ContinueStatement(node)`: Raises `ContinueLoop`.
        -   `visit_WhileStatement(node)` and `visit_ForStatement(node)`: Wrapped their loop body execution in `try-except` blocks to catch `BreakLoop` and `ContinueLoop` exceptions, implementing the desired loop control flow. The `scope_stack` is properly managed (popped) when a `break` or `continue` occurs.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
@@ -94,8 +150,16 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
    - `femcode/examples/loops_advanced.fem`
 ### 8. Error Handling (`try-except`)
- **Description:** Implemented basic error handling mechanisms to gracefully manage runtime errors and prevent program crashes.
+- **Description:** Implemented basic error handling mechanisms to gracefully manage runtime errors and prevent program crashes. This allows Femcode programs to define blocks of code that can "catch" and respond to errors.
 - **Keywords/Syntax:** `Twink` (try block), `Bimboy` (except block)
 - **Technical Details:**
    - **`lexer.py`:** Added regex patterns to recognize `Twink` and `Bimboy` keywords, tokenizing them as `TRY` and `EXCEPT` respectively. These patterns were added to the `token_patterns` list.
    - **`parser.py`:**
        -   New AST node: `TryExceptStatement` (stores the `try_block` and `except_block`).
        -   `parse_statement()`: Modified to recognize `TRY` and dispatch to `parse_try_except_statement()`. 
        -   `parse_try_except_statement()`: Parses the `TRY` keyword, `FEMBOYCORE`, the `try_block` (a `Block` of statements), `PERIODT`, then `EXCEPT`, `FEMBOYCORE`, the `except_block` (another `Block` of statements), and `PERIODT`. It ensures proper block termination.
    - **`interpreter.py`:**
        -   `visit_TryExceptStatement(node)`: Wraps the execution of `node.try_block` in a Python `try-except Exception` block. If any Python `Exception` occurs during the execution of the `try_block`, it catches the exception and then executes `node.except_block`. This provides a basic catch-all error handling mechanism.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
@@ -104,8 +168,13 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
 ### 9. Syntactic Sugar
 #### Compound Assignment Operators
- **Description:** Added shorthand operators for common arithmetic assignments, improving code conciseness.
+- **Description:** Added shorthand operators for common arithmetic assignments, improving code conciseness and readability.
 - **Keywords/Syntax:** `+=`, `-=`, `*=`, `/=`
 - **Technical Details:**
    - **`lexer.py`:** Added token patterns for `PLUS_ASSIGN`, `MINUS_ASSIGN`, `MUL_ASSIGN`, and `DIV_ASSIGN`. These are matched as two-character operators and placed before single-character operators in `token_patterns` to ensure correct precedence.
    - **`parser.py`:**
        -   `parse_assignment_statement()`: Modified to check for these new compound assignment token types (`PLUS_ASSIGN`, `MINUS_ASSIGN`, etc.). If a compound assignment is found, it transforms it into a standard `Assign` AST node where the right-hand side is a `BinOp` representing the equivalent arithmetic operation (e.g., `count += 3` is parsed as `count = count + 3`). This simplifies interpreter logic.
    - **`interpreter.py`:** The `visit_Assign` method implicitly handles the compound assignments because the parser transforms them into standard `Assign` and `BinOp` nodes, requiring no special handling in the interpreter itself.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
@@ -113,8 +182,17 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
    - `femcode/examples/syntactic_sugar.fem`
 #### Increment/Decrement Operators
- **Description:** Introduced operators for easily increasing or decreasing numerical variables by one.
+- **Description:** Introduced operators for easily increasing or decreasing numerical variables by one, providing a more concise syntax for common operations.
 - **Keywords/Syntax:** `++`, `--`
 - **Technical Details:**
    - **`lexer.py`:** Added token patterns for `INCREMENT` (`++`) and `DECREMENT` (`--`). These are matched as two-character operators and placed before single-character operators in `token_patterns` to ensure correct precedence.
    - **`parser.py`:**
        -   New AST nodes: `Increment` and `Decrement` (each storing the `Variable` node to be modified).
        -   `factor()` method: Modified to recognize `INCREMENT` and `DECREMENT` tokens immediately following an `ID` (variable name). It creates the corresponding `Increment` or `Decrement` AST node.
        -   `parse_statement()`: Modified to handle `Increment` and `Decrement` AST nodes as standalone statements.
    - **`interpreter.py`:**
        -   `visit_Increment(node)`: Retrieves the current value of the variable (from `node.var_name`), increments it by 1, and updates the variable in the current scope. Includes type checking to ensure the variable is numeric.
        -   `visit_Decrement(node)`: Retrieves the current value of the variable, decrements it by 1, and updates the variable in the current scope. Includes type checking for numeric values.
 - **Files Modified:**
    - `femcode/src/lexer.py`
    - `femcode/src/parser.py`
@@ -122,6 +200,6 @@ This project, `femboy-coding-language`, is located at the root directory `/home/
    - `femcode/examples/syntactic_sugar.fem`
 ### 10. Documentation Updates
- **Description:** Continuously updated the `README.md` documentation to accurately reflect all new features, syntax, and usage instructions, including the `femterpreter` script.
+- **Description:** Continuously updated the `README.md` documentation to accurately reflect all new features, syntax, and usage instructions, including the `femterpreter` script. This ensures that users have up-to-date information on how to use the language.
 - **Files Modified:**
-    - `femcode/docs/README.md`
+    - `femcode/docs/README.md`