Custom Syntax

Custom syntax is the foundation for Jiang’s All-in-one direction. Users can extend Jiang with their ideal domain syntax while keeping Jiang’s built-in type checking and systems-level runtime efficiency.

It lets a package parse a block of source at a specific syntax position, then return a Jiang syntax tree for normal name resolution, type checking, and code generation. It is intended for domain syntaxes such as queries, user interfaces, shaders, protocols, and low-level assembly blocks.

The current implementation supports block invocation:

User user = #sql {
    select * from User where id == \(id)
};

The sql name in #sql is not a normal value name and is not found through import. It usually comes from the current package manifest’s dependency aliases. The compiler can also provide builtin syntax blocks, such as #asm.

Representative Scenarios

The following snippets are sketches that show the kinds of problems custom syntax is meant to solve. The exact syntax is defined by the corresponding lang package.

Object-relational queries can check tables, fields, and interpolated parameters together:

User[] users = #query {
    from User
    where age >= \(min_age)
    select id, name, email
};

User interface templates can keep an HTML-like shape while returning components, attributes, and bindings to Jiang’s type system:

View profile = #html {
    <section class="profile">
        <h1>{user.name}</h1>
        <button on_click={save(user.id)}>Save</button>
    </section>
};

Declarative user interfaces can use a tree-shaped syntax similar to SwiftUI, while the lang package checks component parameters, state bindings, and event signatures:

View settings = #ui {
    VStack(spacing = 12) {
        Text("Settings").font(.title)
        Toggle("Enable cache", is_on = bind(settings.cache_enabled))
        Button("Save") {
            save(settings)
        }
    }
};

Shaders can use graphics-domain syntax, while the lang package generates checked host-side entry points and resource bindings:

Shader blur = #shader {
    texture source: Texture2D<Float4>;
    uniform radius: Float;

    fragment Float4 main(Float2 uv) {
        return sample_blur(source, uv, radius);
    }
};

Vector computation can express batch math clearly, letting the lang package choose vector instructions or a normal backend path:

Vec4 result = #simd {
    let a = load4(lhs);
    let b = load4(rhs);
    fma(a, b, bias)
};

Binary protocols can describe field numbers, byte order, length relationships, and optional fields in one place, then generate efficient encoders and decoders:

Codec<User> user_codec = #binary {
    message User {
        1: UInt64 id;
        2: UInt8[]& name;
        3: UInt8[]&? email;
    }
};

State machines and protocol flows can spell out transitions so the lang package can check missing branches and illegal moves:

StateMachine connection = #state {
    disconnected -> connecting on connect;
    connecting -> connected on ready;
    connected -> disconnected on close;
};

Command-line interfaces can be declared once, then generate parsing, help text, and a typed options object:

CliOptions options = #cli {
    command build {
        flag release: Bool = false;
        option target: Target = native;
        arg input: Path;
    }
};

Service routes can keep path parameters, request bodies, and return values together while preserving static types for handlers:

Router api = #route {
    GET /users/:id -> get_user(id: UserId) -> User;
    POST /users -> create_user(body: NewUser) -> User;
};

Data pipelines can spell out steps and dependencies so the lang package can check whether each stage’s output matches the next stage’s input:

Pipeline image_pipeline = #pipeline {
    load(path)
        |> decode_png
        |> resize(width = 256, height = 256)
        |> encode_webp(quality = 90)
};

Registers and bit fields can mirror a hardware manual, then generate low-level access code:

RegisterBlock uart = #mmio {
    base 0x4000_1000;
    reg control: UInt32 offset 0x00 {
        enable: bit 0 read_write;
        mode: bits 1..2 read_write;
    }
    reg status: UInt32 offset 0x04 read_only;
};

Builtin Assembly Blocks

#asm is a builtin syntax block. It is meant for small pieces of target-specific assembly inside Jiang functions, such as system calls, startup code, special instructions, or rare backend boundary cases.

Int linux_syscall1(Int number, Int arg0) {
    return #asm {
        result Int;
        code "syscall";
        in "rax" number;
        in "rdi" arg0;
        out "rax";
        clobber "rcx";
        clobber "r11";
        memory;
        volatile;
    };
}

An assembly block can also be used as a statement. If no value is needed, omit result:

Void fence() {
    #asm {
        code "mfence";
        memory;
        volatile;
    }
}

#asm is the short name. A user lang package can override that short name. To explicitly use the compiler builtin, write the full builtin path:

Int linux_syscall1_builtin(Int number, Int arg0) {
    return #jiang.asm {
        result Int;
        code "syscall";
        in "rax" number;
        in "rdi" arg0;
        out "rax";
        clobber "rcx";
        clobber "r11";
        memory;
        volatile;
    };
}

The current #asm block supports:

• result T;: the result type. If omitted, the block returns Void.
• code "...";: the assembly text. This item is required.
• in "register-or-constraint" value;: an input operand.
• out "register-or-constraint";: an output operand.
• clobber "register-or-resource";: a register or resource modified by the assembly.
• memory;: the assembly may read or write ordinary memory.
• volatile;: the assembly has side effects and should not be freely removed or reordered.
• noreturn;: the assembly does not return.

Assembly syntax is target-specific. Names such as rax, rdi, and syscall are only meaningful on specific processors and operating systems. Most application code should not use #asm; it is mainly for runtime code, system boundaries, and narrow platform-specific operations.

Documentation Blocks

#doc attaches Markdown source to the declaration that follows it. Text on the header line creates a single-line document:

#doc Returns the sum of two integers.
public Int add(Int left, Int right) {
    return left + right;
}

If the header line has no text, an otherwise standalone #end ends the document. Use #doc(module) before the first declaration other than an import declaration (including before an alias) to document the current module:

#doc(module) Collection utilities.

#doc
    ## Vector

    Stores elements in contiguous memory.
#end
public struct Vector<T> {
}

A lang dependency may override the short name #doc; use #jiang.doc to select the builtin provider explicitly. Documentation is available to reflection and documentation generators.

Whole-File Integration

Custom syntax also supports whole source files. Import a non-.jiang file through a configured extension mapping; the matching Lang provider parses it into a Jiang module:

alias schema = import "user.proto";
alias layout = import "dashboard.ui";
alias kernel = import "blur.shader";

User user = schema.User(id = 1, name = "Jiang");
View page = layout.render(user);

External language files are not a separate code-generation step; they are part of the package dependency graph. Fields, functions, resources, and generated types can still participate in Jiang name resolution, type checking, and backend optimization.

Embedded Scripting

Jiang itself is designed around ownership and lifetimes; it does not require garbage collection by default. If Jiang later introduces garbage collection, it should be an optional runtime capability chosen by users and packages for the cases that need it. On that optional runtime, custom syntax can host scripting languages. Script blocks or script files could become part of a Jiang package, useful for game logic, plugins, hot-update rules, and user automation:

Script startup = #script {
    let player = world.find("player")
    player.inventory.add("key")
    on event.dialog_finished {
        world.open_door("north_gate")
    }
};

The script side can use a more dynamic object model, while the host side still uses Jiang to define resources, boundary types, and call interfaces. That gives scripts a flexible surface language while keeping performance-critical paths in Jiang’s ownership and lifetime model.

Using a Lang Package

The using package declares a dependency:

#package {
    name = "app";
    root = "src/main.jiang";
    dependencies {
        sql = "../sql_lang";
    }
}

The dependency must be a lang package:

#package {
    name = "sql_lang";
    root = "lang.jiang";
    type = .lang;
}

Mark a concrete, zero-argument-constructible provider type in the package root with @entry(lang). It must implement std.jiang.syntax.Provider; its name is arbitrary and it may be private.

Provider Entry Points

A provider implements std.jiang.syntax.Provider. Providers using Jiang’s default lexical rules only need to implement parse:

import std;

@entry(lang)
struct Provider: std.jiang.syntax.Provider {
    public std.jiang.syntax.Ast parse(
        Self&! self,
        std.jiang.syntax.Input input,
        std.jiang.syntax.SyntaxContext<std.jiang.syntax.EmptyLangContext>& syntax
    ) {
        _ parser! = std.jiang.syntax.default_parser(syntax.syntax, input);
        std.jiang.syntax.Expr value = parser.int_literal(input.name_span, "0");
        return parser.ast(value);
    }
}

The default scan recognizes nested delimiters, strings, comments, and end of file. A provider with custom tokens can override scan, store Token<CustomKind> values in its own instance, and construct Parser<CustomKind> in parse.

Parser methods create typed expression, statement, declaration, type, pattern, and attribute handles. parser.ast(handle) returns the result. Choose a role that fits the position: expressions in expressions, declarations at the top level, and members inside a type.

Language Aliases and File Extensions

Register the dependency once, then map a language alias to it:

#package {
    dependencies {
        tools = "../schema-tools";
    }
    lang schema {
        package = tools;
        extensions = ["schema", "model"];
    }
}

Use #schema { ... } or import models.schema. The provider package may declare default extensions in lang { extensions = ["schema", "model"]; }; the using package replaces that list with its own mapping. Without an explicit extension list, the language alias is the default extension. .jiang retains native Jiang syntax. Whole-file providers receive the .none delimiter and return declarations. The file can also be the package root. Its contents and provider dependencies participate in compiler caching.

A package can provide both Lang and generation entries. An explicit same-named generate configuration takes precedence when selecting a generator by language alias.

Shared Keywords and Context

A stateless Provider can use the default EmptyLangContext and implement only parse. To share keywords or other data, declare LangContext, initialize it with create_context(Session&), and read it through context.lang. One language shares its context within a compilation; separate compilations are isolated. Each block still has its own Provider instance. Keep block tokens and temporary parsing state out of shared context.

This language accepts both #value { value 42 } and #value { 42 }:

import std;

public struct Keywords: Copyable {
    std.jiang.syntax.SymbolId value;
}

@entry(lang)
struct ValueProvider: std.jiang.syntax.Provider {
    public associated LangContext = Keywords;

    @life()
    public Keywords create_context(std.jiang.syntax.Session& session) {
        return Keywords(value = session.intern("value"));
    }

    public std.jiang.syntax.Ast parse(
        Self&! self, std.jiang.syntax.Input input,
        std.jiang.syntax.SyntaxContext<Keywords>& context
    ) {
        _ parser! = std.jiang.syntax.default_parser(context.syntax, input);
        parser.match_keyword(context.lang.value);
        _ value = parser.parse_expression();
        return parser.ast(value);
    }
}

session.intern() returns IDs compatible with default tokens, so keywords need not be registered for every block.

Nested Languages and Attributes

At a RawBlock, call parser.parse_raw_block() to obtain Ast?. Inspect role() or use as_attribute(), as_decl(), as_member() and the other typed accessors. Your language can therefore accept #doc without recognizing its name or implementing Markdown parsing.

The receiving language chooses the target and attaches attributes to a Decl or Member with with_attributes. Keep pending attributes in parser.list<Attribute>() until the next target arrives. Report dangling attributes at the end of a block; do not attach an inner block’s documentation to an outer declaration. Package configuration fields may instead ignore attributes.

parser.ast(member) returns one member. Use parser.members(span, values) for member sequences and parser.declarations(span, values) for declaration sequences. Empty and single-element sequences remain sequences; read them with parser.len(sequence) and parser.at(sequence, index).

If the current token is not a RawBlock, expansion reports a diagnostic and returns null without consuming it. A failed RawBlock expansion consumes that token and keeps its diagnostics. peek and advance only read tokens; they do not expand language blocks.

A custom tokenizer can call scan_raw_block() after begin_token(), then retain the returned block ID and original span. Expand that saved default token with parser.parse_raw_block(raw); this overload does not advance the custom parser’s cursor. Consume your own custom token explicitly, and do not fabricate identities or reuse them across source files.

Lists and Backtracking

Create syntax-handle lists with parser.list<T>(), then use append, len and at. AST construction accepts managed lists alongside existing slice arguments. These lists hold supported syntax handles, not arbitrary application values.

Inside parse, with parser and input already available:

_ statements = parser.list<std.jiang.syntax.Stmt>();
_ span = input.name_span;
_ value = parser.int_literal(span, "42");
statements.append(parser.return_statement(span, value));
_ body = parser.block(span, statements);

Save parser.checkpoint() before a speculative parse. Use parser.rewind(saved) to try another interpretation. Rewind restores tokens, diagnostics, AST changes and managed lists, including attributes attached to existing nodes. Handles created after the checkpoint become invalid; earlier lists regain their previous contents. On success, continue without a separate commit operation.

Provider fields, ordinary vectors and external side effects are not rolled back. Put pending attributes in managed lists when they must rewind. Default scanned tokens support repeated parsing; custom tokens must likewise remain readable if your parser retries them.

Default tokens use single_quoted/double_quoted to distinguish quote forms. SymbolId contains decoded text; use the span to read original quotes and escapes. Constructing a Jiang character AST requires exactly one Unicode scalar.