Use cargo make

CONTRIBUTING.md

@@ -5,8 +5,8 @@ We ask that _before_ opening a PR however, you check to see if there is an issue
 wish to make. If there isn't, it's best to open a new issue first to discuss it, to save you time in future
 and help us further ascertain the crux of the issue. If an issue already exists, please add to the discussion there.
 
-Once an issue has been discussed and agreement that it should be acted upon, if you wish to work on a PR
-to address it, please assign the issue to yourself, so that others know that it is being worked on.
+Once an issue has been discussed and agreement reached that it should be acted upon, if you wish to work on a PR
+to address it, please assign the issue to yourself, so that others know that you're working on it.
 
 ## Sign the Contributor License Agreement
 
@@ -15,31 +15,103 @@ before we can accept pull requests from you.
 
 ## Development
 
-The following information may help with contributing to this project. The workspace contains two packages:
+The following information will help in getting up and running:
 
-### api_generator
+### Prerequisites
 
-A small executable to download REST API specs from GitHub and generate much of the client from the specs. Run with
+The project makes use of the following, which should be installed
+
+- [**Docker**](https://www.docker.com/)
+
+  Docker is used to start instances of Elasticsearch by using 
+  [Elastic's Elasticsearch docker images](https://container-library.elastic.co/).
+  For Windows, use [Docker with WSL 2 backend](https://docs.docker.com/docker-for-windows/wsl/).
+
+- [**Cargo make**](https://sagiegurari.github.io/cargo-make/)
+
+  Cargo make is used to define and configure a set of tasks, and run them as a flow. This helps with performing actions 
+  such as starting an Elasticsearch instance for integration tests
+
+  Cargo make can be installed with
+
+  ```sh
+  cargo install --force cargo-make
+  ```
+
+### Cargo make
+
+Cargo make is used to define and configure a set of tasks, and run them as a flow. To see all of the tasks defined
 
 ```sh
-cargo run -p api_generator
+cargo make --list-all-steps
 ```
 
-from the repository root directory, and follow the prompts. The minimum REST API spec version compatible with the 
-generator is `v7.4.0`.
+The `Elasticsearch` category of steps are specifically defined for this project and are defined in
+[Makefile.toml](Makefile.toml).
 
-The api_generator makes heavy use of the [`syn`](https://docs.rs/syn/1.0.5/syn/) and [`quote`](https://docs.rs/quote/1.0.2/quote/) crates to generate Rust code from the REST API specs.
-The `quote!` macro is particularly useful as it accepts Rust code that can include placeholder tokens (prefixed with `#`)
-that will be interpolated during expansion. Unlike procedural macros, the token stream returned by the `quote!` macro
-can be `to_string()`'ed and written to disk, and this is used to create much of the client scaffolding.
+- Build all packages
+
+  ```sh
+  cargo make build
+  ```
+
+- Generate client from REST specs
+
+  ```sh
+  cargo make generate-api
+  ```
+
+- Run Elasticsearch package tests
+
+  Optionally pass 
+
+  - `STACK_VERSION`: Elasticsearch version like `7.9.0` or can be
+  a snapshot release like `7.x-SNAPSHOT`
+
+  ```sh
+  cargo make test --env STACK_VERSION=<e.g. 7.9.0>
+  ```
+
+- Run YAML tests
+
+  Optionally pass 
 
-### elasticsearch
+  - `STACK_VERSION`: Elasticsearch version like `7.9.0` or can be
+  a snapshot release like `7.x-SNAPSHOT`
+  - `TEST_SUITE`: Elasticsearch distribution of `oss` or `xpack`
+
+  ```sh
+  cargo make test-yaml --env STACK_VERSION=<e.g. 7.9.0> --env TEST_SUITE=<xpack or oss>
+  ```
 
-The client package crate. The client exposes all Elasticsearch APIs as associated functions, either on
+### Packages
+
+The workspace contains the following packages:
+
+- #### `elasticsearch`
+
+  The client package crate. The client exposes all Elasticsearch APIs as associated functions, either on
 the root client, `Elasticsearch`, or on one of the _namespaced clients_, such as `Cat`, `Indices`, etc. The _namespaced clients_
 are based on the grouping of APIs within the [Elasticsearch](https://github.com/elastic/elasticsearch/tree/master/rest-api-spec) and [X-Pack](https://github.com/elastic/elasticsearch/tree/master/x-pack/plugin/src/test/resources/rest-api-spec/api) REST API specs from which much of the client is generated.
 All API functions are `async` only, and can be `await`ed.
 
+- #### `api_generator` 
+
+  A small executable that downloads REST API specs from GitHub and generates much of the client package from the specs. 
+The minimum REST API spec version compatible with the generator is `v7.4.0`.
+
+  The `api_generator` package makes heavy use of the [`syn`](https://docs.rs/syn/1.0.5/syn/) and [`quote`](https://docs.rs/quote/1.0.2/quote/) crates to generate Rust code from the REST API specs.
+The `quote!` macro is particularly useful as it accepts Rust code that can include placeholder tokens (prefixed with `#`)
+that will be interpolated during expansion. Unlike procedural macros, the token stream returned by the `quote!` macro
+can be `to_string()`'ed and written to disk, and this is used to create much of the client scaffolding.
+
+- #### `yaml_test_runner`
+
+  A small executable that downloads YAML tests from GitHub and generates client tests from the YAML tests. The
+  version of YAML tests to download are determined from the commit hash of a running Elasticsearch instance.
+
+  The `yaml_test_runner` package can be run with `cargo test` to run the generated client tests.
+
 ### Design principles
 
 1. Generate as much of the client as feasible from the REST API specs
@@ -75,18 +147,7 @@ The required toolchain for packages in the workspace are controlled
 by a `rust-toolchain` file in the root of each package.
 
 `elasticsearch` package compiles and runs with rust stable. 
-
-`elasticsearch` tests incorporate testing the examples
-given in the README.md file, using the
-[`external_doc`](https://doc.rust-lang.org/unstable-book/language-features/external-doc.html)
-experimental feature. To run these tests too, run with cargo nightly,
-using the `+<toolchain>` command line override
-
-```sh
- cargo +nightly test -p elasticsearch --doc
-```
-
-`api_generator` package requires rust nightly.
+`api_generator` and `yaml_test_runner` packages require rust nightly.
 
 ### Coding style guide
 
@@ -99,7 +160,7 @@ Rust code can be formatted using [`rustfmt`](https://github.com/rust-lang/rustfm
 To format all packages in a workspace, from the workspace root
 
 ```sh
-cargo fmt
+cargo make format
 ```
 
 It is strongly recommended to run this before opening a PR.
@@ -111,7 +172,7 @@ It is strongly recommended to run this before opening a PR.
 Run clippy before opening a PR
 
 ```sh
-cargo clippy
+cargo make clippy
 ```
 
 ### Running MSVC debugger in VS Code

Makefile.toml

@@ -0,0 +1,149 @@
+[config]
+default_to_workspace = false
+
+[env]
+# Determines the version of Elasticsearch docker container used
+STACK_VERSION = "8.0.0-SNAPSHOT"
+# Determines the distribution of docker container used. Either xpack or oss
+TEST_SUITE = "xpack"
+
+[tasks.set-oss-env]
+category = "Elasticsearch"
+description = "Sets ELASTICSEARCH_URL environment variable for later tasks when oss test suite used"
+private = true
+condition = { env = { "TEST_SUITE" = "oss" } }
+env = { "ELASTICSEARCH_URL" = "http://localhost:9200" }
+
+[tasks.set-xpack-env]
+category = "Elasticsearch"
+description = "Sets ELASTICSEARCH_URL environment variable for later tasks when xpack test suite used"
+private = true
+condition = { env = { "TEST_SUITE" = "xpack" } }
+env = { "ELASTICSEARCH_URL" = "https://elastic:changeme@localhost:9200" }
+
+[tasks.run-yaml-test-runner]
+category = "Elasticsearch"
+description = '''
+Runs yaml_test_runner crate to generate tests from yaml files for a given Elasticsearch commit.
+The commit to use is retrieved from the running Elasticsearch instance
+'''
+private = true
+script = ["cargo run -p yaml_test_runner -- -u ${ELASTICSEARCH_URL}"]
+dependencies = ["start-elasticsearch"]
+
+[tasks.run-yaml-test-runner.windows]
+script = ["cargo run -p yaml_test_runner -- -u %ELASTICSEARCH_URL%"]
+
+[tasks.test-yaml-test-runner]
+category = "Elasticsearch"
+private = true
+condition = { env_set = [ "ELASTICSEARCH_URL" ] }
+env = { "ES_TEST_SERVER" = "${ELASTICSEARCH_URL}" }
+command = "cargo"
+args = ["test", "-p", "yaml_test_runner", "--", "--test-threads=1"]
+dependencies = ["generate-yaml-tests"]
+
+[tasks.test-elasticsearch]
+category = "Elasticsearch"
+private = true
+condition = { env_set = [ "ELASTICSEARCH_URL" ], env = { "TEST_SUITE" = "xpack" } }
+env = { "ES_TEST_SERVER" = "${ELASTICSEARCH_URL}" }
+command = "cargo"
+args = ["test", "-p", "elasticsearch"]
+dependencies = ["start-elasticsearch"]
+
+[tasks.run-api-generator]
+category = "Elasticsearch"
+private = true
+command = "cargo"
+args = ["run", "-p", "api_generator"]
+
+# ============
+# Public tasks
+# ============
+
+[tasks.start-elasticsearch]
+category = "Elasticsearch"
+description = "Starts Elasticsearch docker container with the given version and distribution"
+condition = { env_set = [ "STACK_VERSION", "TEST_SUITE" ] }
+script = ["DETACH=true bash .ci/run-elasticsearch.sh"]
+dependencies = ["set-oss-env", "set-xpack-env"]
+
+[tasks.start-elasticsearch.windows]
+script = ["bash -c \"STACK_VERSION=%STACK_VERSION% TEST_SUITE=%TEST_SUITE% DETACH=true bash .ci/run-elasticsearch.sh\""]
+
+[tasks.stop-elasticsearch]
+category = "Elasticsearch"
+description = "Stops Elasticsearch docker container, if running"
+condition = { env_set = [ "STACK_VERSION", "TEST_SUITE" ] }
+script = ["CLEANUP=true bash .ci/run-elasticsearch.sh"]
+dependencies = ["set-oss-env", "set-xpack-env"]
+
+[tasks.stop-elasticsearch.windows]
+script = ["bash -c \"STACK_VERSION=%STACK_VERSION% TEST_SUITE=%TEST_SUITE% CLEANUP=true bash .ci/run-elasticsearch.sh\""]
+
+[tasks.test-yaml]
+category = "Elasticsearch"
+description = "Generates and runs yaml_test_runner crate xpack/oss tests against a given Elasticsearch version"
+condition = { env_set = [ "STACK_VERSION", "TEST_SUITE" ] }
+dependencies = ["generate-yaml-tests", "test-yaml-test-runner"]
+run_task = "stop-elasticsearch"
+
+[tasks.test]
+category = "Elasticsearch"
+clear = true
+description = "Runs Elasticsearch crate tests against a given Elasticsearch version"
+env = { "TEST_SUITE" = { value = "xpack", condition = { env_set = ["TEST_SUITE"] } } }
+dependencies = ["test-elasticsearch"]
+run_task = "stop-elasticsearch"
+
+[tasks.generate-yaml-tests]
+category = "Elasticsearch"
+description = "Generates Elasticsearch client tests from YAML tests"
+dependencies = ["run-yaml-test-runner"]
+run_task = "format"
+
+[tasks.generate-api]
+category = "Elasticsearch"
+description = "Generates Elasticsearch client from REST API specs"
+dependencies = ["run-api-generator"]
+run_task = "format"
+
+
+[tasks.default]
+clear = true
+script = ['''
+    echo
+    echo "Main tasks:"
+    echo "- generate-api: Generates Elasticsearch client from REST API specs"
+    echo "- start-elasticsearch: Starts Elasticsearch docker container with the given version and distribution"
+    echo "- stop-elasticsearch: Stops Elasticsearch docker container, if running"
+    echo "- test-yaml: Generates and runs yaml_test_runner crate xpack/oss tests against a given Elasticsearch version"
+    echo "- test: Runs Elasticsearch crate tests against a given Elasticsearch version"
+    echo
+    echo "Most tasks use these environment variables:"
+    echo "- STACK_VERSION (default '$STACK_VERSION'): the version of Elasticsearch"
+    echo "- TEST_SUITE ('oss' or 'xpack', default '$TEST_SUITE'): the distribution of Elasticsearch"
+    echo
+    echo "Run 'cargo make --list-all-steps' for a complete list of available tasks."
+    echo
+''']
+
+[tasks.default.windows]
+script = ['''
+    @echo off
+    echo.
+    echo Main tasks:
+    echo - generate-api: Generates Elasticsearch client from REST API specs
+    echo - start-elasticsearch: Starts Elasticsearch docker container with the given version and distribution
+    echo - stop-elasticsearch: Stops Elasticsearch docker container, if running
+    echo - test-yaml: Generates and runs yaml_test_runner crate xpack/oss tests against a given Elasticsearch version
+    echo - test: Runs Elasticsearch crate tests against a given Elasticsearch version
+    echo.
+    echo Most tasks use these environment variables:
+    echo - STACK_VERSION (default '$STACK_VERSION'): the version of Elasticsearch
+    echo - TEST_SUITE ('oss' or 'xpack', default '$TEST_SUITE'): the distribution of Elasticsearch
+    echo.
+    echo Run 'cargo make --list-all-steps' for a complete list of available tasks.
+    echo.
+''']

api_generator/Cargo.toml

@@ -20,7 +20,6 @@ quote = "~0.3"
 reduce = "0.1.2"
 regex = "1.3.1"
 reqwest = "~0.9"
-rustfmt-nightly = "1.4.19"
 semver = "0.9.0"
 serde = "~1"
 serde_json = "~1"

api_generator/src/generator/code_gen/mod.rs

@@ -31,8 +31,9 @@ use std::str;
 pub fn use_declarations() -> Tokens {
     quote!(
         #![allow(unused_imports)]
+
         use crate::{
-            client::{Elasticsearch},
+            client::Elasticsearch,
             params::*,
             error::Error,
             http::{

api_generator/src/generator/code_gen/namespace_clients.rs

@@ -103,7 +103,7 @@ pub fn generate(api: &Api, docs_dir: &PathBuf) -> Result<Vec<(String, String)>,
             }
         ));
 
-        let generated = rust_fmt(tokens.to_string())?;
+        let generated = tokens.to_string();
         output.push((namespace.to_string(), generated));
     }
 

api_generator/src/generator/code_gen/params.rs

@@ -29,7 +29,8 @@ pub fn generate(api: &Api) -> Result<String, failure::Error> {
         generate_param(&mut tokens, &e);
     }
 
-    rust_fmt(tokens.to_string())
+    let generated = tokens.to_string();
+    Ok(generated)
 }
 
 fn generate_param(tokens: &mut Tokens, e: &ApiEnum) {

api_generator/src/generator/code_gen/root.rs

@@ -56,6 +56,6 @@ pub fn generate(api: &Api, docs_dir: &PathBuf) -> Result<String, failure::Error>
         }
     ));
 
-    let generated = rust_fmt(tokens.to_string())?;
+    let generated = tokens.to_string();
     Ok(generated)
 }