Skip to content

Add initial Ruby support #270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 43 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
41047cb
Add skeleton Ruby project
hendrikvanantwerpen May 11, 2023
9731142
Add storage and visualization features as required features
rewinfrey May 11, 2023
ae49472
Add a literal and statement test
rewinfrey May 11, 2023
4ad6b0a
Scaffold module / class include test
rewinfrey May 12, 2023
7fc44fa
Use working example.rb file
rewinfrey May 12, 2023
a97f2e9
Revert "Add storage and visualization features as required features"
hendrikvanantwerpen May 12, 2023
95934bb
Remove empty LICENSE file
hendrikvanantwerpen May 12, 2023
afd5f1a
Require clap derive explicitly
hendrikvanantwerpen May 12, 2023
86366c9
Use Ruby comment syntax
hendrikvanantwerpen May 12, 2023
24a61c7
Make includes module test more interesting
rewinfrey May 12, 2023
9c7065e
Fist class def test
hendrikvanantwerpen May 15, 2023
9d0dead
Skip unsupported tests
hendrikvanantwerpen May 15, 2023
e5ff578
Support class variables
hendrikvanantwerpen May 15, 2023
757c895
Merge remote-tracking branch 'origin/main' into rubies-are-forever
hendrikvanantwerpen May 16, 2023
bfa742d
Merge remote-tracking branch 'origin/main' into rubies-are-forever
hendrikvanantwerpen May 16, 2023
213f5a5
Add nested class test
hendrikvanantwerpen May 16, 2023
93c00bb
Add edge from program lexical scope to the root node
rewinfrey May 16, 2023
e4eb597
Differentiate between class definitions using scoped names versus sin…
rewinfrey May 16, 2023
738541f
Add support for resolving ::Foo scoped constants
rewinfrey May 16, 2023
29ce3c8
Add convenience scripts for development
hendrikvanantwerpen May 17, 2023
6f75947
Reorganize program to global scope connectivity
hendrikvanantwerpen May 17, 2023
9e05d34
A bit of header styling
hendrikvanantwerpen May 17, 2023
cc6cbf5
Factor out identifiers into seperate rules
hendrikvanantwerpen May 17, 2023
e4639c5
Support root variable definitions
hendrikvanantwerpen May 17, 2023
3d4321c
Always use same output file
hendrikvanantwerpen May 17, 2023
8bcf821
Support visiblity between split class definitions
hendrikvanantwerpen May 17, 2023
5affcd5
Add module support
hendrikvanantwerpen May 17, 2023
fc63478
Add push / pop node resolution for `new` method calls on class constants
rewinfrey May 17, 2023
a68be21
Model class instantiation with `new` constructor method call
rewinfrey May 17, 2023
455e90c
Introduce static_defs and instance_defs scoped variables.
rewinfrey May 19, 2023
d9e7f52
Update tests to use consistent commenting style and additional assert…
rewinfrey May 19, 2023
d180b11
Remove `.lexical_defs`
hendrikvanantwerpen May 22, 2023
d5f310c
Use `.{def,ref}_value`
hendrikvanantwerpen May 22, 2023
ee48789
Implement local variables and lexical scoping
hendrikvanantwerpen May 22, 2023
22812c9
Also allow dereferencing `::` through constants
hendrikvanantwerpen May 22, 2023
2ef2114
Implement refinement
hendrikvanantwerpen May 23, 2023
a7c61a4
Prefer local definitions over surrounding context
hendrikvanantwerpen May 23, 2023
a347fd1
Use tree-sitter query predicates instead of conditional statements to…
hendrikvanantwerpen May 23, 2023
4218062
Add skipped test that triggers resolution bug
hendrikvanantwerpen May 24, 2023
db447f1
Prevent path searching within refinement subgraphs when calling conte…
rewinfrey May 24, 2023
83518c9
Merge remote-tracking branch 'origin/main' into rubies-are-forever
hendrikvanantwerpen May 30, 2023
df77c87
Merge remote-tracking branch 'origin/main' into rubies-are-forever
hendrikvanantwerpen Jun 8, 2023
8aeff0b
Merge remote-tracking branch 'origin/main' into rubies-are-forever
hendrikvanantwerpen Oct 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/workflows/publish-tree-sitter-stack-graphs-ruby.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Publish tree-sitter-stack-graphs-ruby release

on:
push:
tags:
- tree-sitter-stack-graphs-ruby-v*

jobs:
publish-crate:
runs-on: ubuntu-latest
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
CRATE_DIR: './languages/tree-sitter-stack-graphs-ruby'
steps:
- name: Install Rust environment
uses: hecrj/setup-rust-action@v1
- name: Checkout repository
uses: actions/checkout@v3
# TODO Verify the crate version matches the tag
- name: Test crate
run: cargo test --all-features
working-directory: ${{ env.CRATE_DIR }}
- name: Verify publish crate
run: cargo publish --dry-run
working-directory: ${{ env.CRATE_DIR }}
- name: Publish crate
run: cargo publish
working-directory: ${{ env.CRATE_DIR }}
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
create-release:
needs: publish-crate
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Create GitHub release
uses: ncipollo/release-action@v1
with:
body: |
Find more info on all releases at https://crates.io/crates/tree-sitter-stack-graphs-ruby.
token: ${{ secrets.GITHUB_TOKEN }}
3 changes: 3 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*.html
/Cargo.lock
/target
6 changes: 6 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Changelog for tree-sitter-stack-graphs-ruby

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
40 changes: 40 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
[package]
name = "tree-sitter-stack-graphs-ruby"
version = "0.1.0"
description = "Stack graphs definition for Ruby using tree-sitter-ruby"
readme = "README.md"
keywords = ["tree-sitter", "stack-graphs", "ruby"]
authors = [
"GitHub <opensource+stack-graphs@github.com>",
]
license = "MIT OR Apache-2.0"
edition = "2018"

[[bin]]
name = "tree-sitter-stack-graphs-ruby"
path = "rust/bin.rs"
required-features = ["cli"]

[lib]
path = "rust/lib.rs"
test = false

[[test]]
name = "test"
path = "rust/test.rs"
harness = false

[features]
cli = ["anyhow", "clap", "tree-sitter-stack-graphs/cli"]

[dependencies]
anyhow = { version = "1.0", optional = true }
clap = { version = "4", optional = true, features = ["derive"] }
tree-sitter-graph = { version = "0.10.0" }
tree-sitter-stack-graphs = { version = "0.6.0", path = "../../tree-sitter-stack-graphs" }
tree-sitter-ruby = "0.20.0"
uuid = { version = "1.3", features = ["v4", "fast-rng"] }

[dev-dependencies]
anyhow = "1.0"
tree-sitter-stack-graphs = { version = "0.6.0", path = "../../tree-sitter-stack-graphs", features = ["cli"] }
98 changes: 98 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# tree-sitter-stack-graphs definition for Ruby

This project defines tree-sitter-stack-graphs rules for Ruby using the [tree-sitter-ruby][] grammar.

[tree-sitter-ruby]: https://crates.io/crates/tree-sitter-ruby

## Usage

To use this library, add the following to your `Cargo.toml`:

``` toml
[dependencies]
tree-sitter-stack-graphs-ruby = "0.0.1"
```

Check out our [documentation](https://docs.rs/tree-sitter-stack-graphs-ruby/*/) for more details on how to use this library.

## Command-line Program

The command-line program for `tree-sitter-stack-graphs-ruby` lets you do stack graph based analysis and lookup from the command line.

Install the program using `cargo install` as follows:

``` sh
$ cargo install --features cli tree-sitter-stack-graphs-ruby
$ tree-sitter-stack-graphs-ruby --help
```

## Development

The project is written in Rust, and requires a recent version installed. Rust can be installed and updated using [rustup][].

[rustup]: https://rustup.rs/

The project is organized as follows:

- The stack graph rules are defined in `src/stack-graphs.tsg`.
- Builtins sources and configuration are defined in `src/builtins.rb` and `builtins.cfg` respectively.
- Tests are put into the `test` directory.

### Building and Running Tests

Build the project by running:

``` sh
$ cargo build
```

Run the tests as follows:

``` sh
$ cargo test
```

The project consists of a library and a CLI. By default, running `cargo` only applies to the library. To run `cargo` commands on the CLI as well, add `--features cli` or `--all-features`.

Run the CLI from source as follows:

``` sh
$ cargo run --features cli -- ARGS
```

Sources are formatted using the standard Rust formatted, which is applied by running:

``` sh
$ cargo fmt
```

### Writing TSG

The stack graph rules are written in [tree-sitter-graph][]. Checkout the [examples][],
which contain self-contained TSG rules for specific language features. A VSCode
[extension][] is available that provides syntax highlighting for TSG files.

[tree-sitter-graph]: https://github.com/tree-sitter/tree-sitter-graph
[examples]: https://github.com/github/stack-graphs/blob/main/tree-sitter-stack-graphs/examples/
[extension]: https://marketplace.visualstudio.com/items?itemName=tree-sitter.tree-sitter-graph

Parse and test a single file by executing the following commands:

``` sh
$ cargo run --features cli -- parse FILES...
$ cargo run --features cli -- test TESTFILES...
```

Generate a visualization to debug failing tests by passing the `-V` flag:

``` sh
$ cargo run --features cli -- test -V TESTFILES...
```

To generate the visualization regardless of test outcome, execute:

``` sh
$ cargo run --features cli -- test -V --output-mode=always TESTFILES...
```

Go to https://crates.io/crates/tree-sitter-stack-graphs for links to examples and documentation.
5 changes: 5 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/contrib/match
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/usr/bin/env bash

set -eu

cargo run --features cli -- match "$@"
5 changes: 5 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/contrib/parse
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/usr/bin/env bash

set -eu

cargo run --features cli -- parse "$@"
5 changes: 5 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/contrib/test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/usr/bin/env bash

set -eu

cargo run --features cli -- test -V=test.html --output-mode always "$@"
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
36 changes: 36 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/example.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
module A
CONST = 1
def a; end
end

module B
include A

def b; end
end

include B

C = Module.new do
def c; end
def b
puts "c"
end
end

self.extend(C)

class D
include C

def calling_a
puts "calling a: #{a}"
end

def calling_c
puts "calling c: #{c}"
end
end

d = D.new
d.b # => "c"
32 changes: 32 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/rust/bin.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// -*- coding: utf-8 -*-
// ------------------------------------------------------------------------------------------------
// Copyright © 2023, stack-graphs authors.
// Licensed under either of Apache License, Version 2.0, or MIT license, at your option.
// Please see the LICENSE-APACHE or LICENSE-MIT files in this distribution for license details.
// ------------------------------------------------------------------------------------------------

use anyhow::anyhow;
use clap::Parser;
use tree_sitter_stack_graphs::cli::database::default_user_database_path_for_crate;
use tree_sitter_stack_graphs::cli::provided_languages::Subcommands;
use tree_sitter_stack_graphs::NoCancellation;

fn main() -> anyhow::Result<()> {
let lc = match tree_sitter_stack_graphs_ruby::try_language_configuration(&NoCancellation) {
Ok(lc) => lc,
Err(err) => {
eprintln!("{}", err.display_pretty());
return Err(anyhow!("Language configuration error"));
}
};
let cli = Cli::parse();
let default_db_path = default_user_database_path_for_crate(env!("CARGO_PKG_NAME"))?;
cli.subcommand.run(default_db_path, vec![lc])
}

#[derive(Parser)]
#[clap(about, version)]
pub struct Cli {
#[clap(subcommand)]
subcommand: Subcommands,
}
66 changes: 66 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/rust/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
// -*- coding: utf-8 -*-
// ------------------------------------------------------------------------------------------------
// Copyright © 2023, stack-graphs authors.
// Licensed under either of Apache License, Version 2.0, or MIT license, at your option.
// Please see the LICENSE-APACHE or LICENSE-MIT files in this distribution for license details.
// ------------------------------------------------------------------------------------------------

use tree_sitter_stack_graphs::loader::FileAnalyzers;
use tree_sitter_stack_graphs::loader::LanguageConfiguration;
use tree_sitter_stack_graphs::loader::LoadError;
use tree_sitter_stack_graphs::CancellationFlag;

/// The stack graphs tsg source for this language.
pub const STACK_GRAPHS_TSG_PATH: &str = "src/stack-graphs.tsg";
/// The stack graphs tsg source for this language.
pub const STACK_GRAPHS_TSG_SOURCE: &str = include_str!("../src/stack-graphs.tsg");

/// The stack graphs builtins configuration for this language.
pub const STACK_GRAPHS_BUILTINS_CONFIG: &str = include_str!("../src/builtins.cfg");
/// The stack graphs builtins path for this language
pub const STACK_GRAPHS_BUILTINS_PATH: &str = "src/builtins.rb";
/// The stack graphs builtins source for this language.
pub const STACK_GRAPHS_BUILTINS_SOURCE: &str = include_str!("../src/builtins.rb");

/// The name of the file path global variable.
pub const FILE_PATH_VAR: &str = "FILE_PATH";

pub fn language_configuration(cancellation_flag: &dyn CancellationFlag) -> LanguageConfiguration {
try_language_configuration(cancellation_flag).unwrap_or_else(|err| panic!("{}", err))
}

pub fn try_language_configuration(
cancellation_flag: &dyn CancellationFlag,
) -> Result<LanguageConfiguration, LoadError> {
let mut lc = LanguageConfiguration::from_sources(
tree_sitter_ruby::language(),
Some(String::from("source.rb")),
None,
vec![String::from("rb")],
STACK_GRAPHS_TSG_PATH.into(),
STACK_GRAPHS_TSG_SOURCE,
Some((
STACK_GRAPHS_BUILTINS_PATH.into(),
STACK_GRAPHS_BUILTINS_SOURCE,
)),
Some(STACK_GRAPHS_BUILTINS_CONFIG),
FileAnalyzers::new(),
cancellation_flag,
)?;
lc.sgl.functions_mut().add("uuid".into(), UUID);
Ok(lc)
}

struct UUID;

impl tree_sitter_graph::functions::Function for UUID {
fn call(
&self,
_graph: &mut tree_sitter_graph::graph::Graph,
_source: &str,
parameters: &mut dyn tree_sitter_graph::functions::Parameters,
) -> Result<tree_sitter_graph::graph::Value, tree_sitter_graph::ExecutionError> {
parameters.finish()?;
Ok(uuid::Uuid::new_v4().to_string().into())
}
}
23 changes: 23 additions & 0 deletions languages/tree-sitter-stack-graphs-ruby/rust/test.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// -*- coding: utf-8 -*-
// ------------------------------------------------------------------------------------------------
// Copyright © 2023, stack-graphs authors.
// Licensed under either of Apache License, Version 2.0, or MIT license, at your option.
// Please see the LICENSE-APACHE or LICENSE-MIT files in this distribution for license details.
// ------------------------------------------------------------------------------------------------

use anyhow::anyhow;
use std::path::PathBuf;
use tree_sitter_stack_graphs::ci::Tester;
use tree_sitter_stack_graphs::NoCancellation;

fn main() -> anyhow::Result<()> {
let lc = match tree_sitter_stack_graphs_ruby::try_language_configuration(&NoCancellation) {
Ok(lc) => lc,
Err(err) => {
eprintln!("{}", err.display_pretty());
return Err(anyhow!("Language configuration error"));
}
};
let test_path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("test");
Tester::new(vec![lc], vec![test_path]).run()
}
1 change: 1 addition & 0 deletions languages/tree-sitter-stack-graphs-ruby/src/builtins.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[globals]
Empty file.
Loading