[Question] How to properly configure monorepo Typescript project?

[vamcs]

I got tired of serverless-webpack being too complex to set up with a monorepo and decided to try serverless-jetpack instead.

Two questions:

1. Would it work to use jetpack in a monorepo where each service has its own serverless.yml?
2. Could you provide a more complete example with Typescript?

I've read through #74 and started using serverless-scriptable-plugin with a manual build with tsc but I don't quite get what to include or exclude and where to do that.

Additional question: is there any sort of tree shaking going on? If I have the structure below and if I pack each function individually will I only get the (transpiled) get.js file in the get function bundle? Or do I have to explicitly exclude everything else and only include that file for that function?

services/
  service1/
    get.ts
    create.ts

Thanks a lot!

[ryan-roemer]

Hi @vamcs !

We've got lots of clients doing both monorepos + TS with various different setups.

I'd definitely recommend using tracing mode, although there is more configuration involved than legacy mode.

Building TS

Jetpack only reads real JS files that Node.js can read. So you'll want to build your TS and point a serverless handler at that file.

What we normally do in our monorepos is have the root package.json just have a scripts task like build:ts or something that builds everything once, effectively separating the TS build from the serverless packaging. So basically the strategy is "make sure TS is build before running any serverless commands".

This is nice because then serverless package|deploy doesn't have a mandatory build step, which you don't always need. But, if you want it to be part of the lifecycle, the serverless-scriptable-plugin example will probably work fine.

So assuming TS is already built (with a transpiled JavaScript entry point at like services/{NAME}/dist/{ENTRY}.js or something), let's look at ways of serverless package/deploy:

OPTION ONE -- Single Config

This is what we most typically do. Create a root-level serverless.yml config like the following:

service: SERVICE_NAME

custom:
  jetpack:
    concurrency: 4    # If you're on a Multi-CPU machine consider setting this to package in parallel.
    collapsed:
      bail: true      # (OPTIONAL) Abort build if collapsed files in zip. (Sometimes an issue in monorepos).
    preInclude:
      - "!**"         # Start with no files at all.
    trace:            # Trace actual dependencies at each entry point
      dynamic:
        bail: true    # (OPTIONAL) Abort build if untraceable imports.

package:
  individually: true  # Package each function individually

plugins:
  modules:
    - serverless-jetpack

provider: # ...

functions:
  service1:
    # Transpiled JS file at: services/service1/dist/{ENTRY}.js
    handler: services/service1/dist/{ENTRY}.{HANDLER_NAME}

  service2:
    handler: services/service2/dist/{ENTRY}.{HANDLER_NAME}
  
  # ...

To your "tree shaking" comment, this approach won't remove any unused code paths as it doesn't transform any source files. But it does trace and only include the files that are imported anywhere transitively starting at the entry point. In our experience, this ends up being not noticeably larger than an equivalent webpack build and often much nicer for things like debugging because you get stack traces from the real, original source files, etc.

OPTION TWO -- Multiple Configs

You could also place individual config in something like services/{NAME}/serverless.yml. You then wouldn't need to package with package.individually = true because there would be only one function per config.

But you would want to look at serverless config include features to avoid a lot of duplication. And if you wanted things like concurrent builds you'd have to script that yourself.

Assessment

Tracing mode normally needs some extra options for imports that can't be directly inferred, so I've given you a config with two bail = true safety guards to fail the build if potential correctness issues are detected (and some of these things are not detected by webpack, so you may discover new runtime dependencies). In particular, you'll likely need some work with issues like https://github.com/FormidableLabs/serverless-jetpack#handling-dynamic-import-misses

Most configs end up with some combination of:

jetpack:
    concurrency: 2

    trace:
      # Manually add other JS files to trace from to include in a bundle.
      include:
        # ...
      
      # Completely ignore certain imports.
      ignores:
        # ...
        # Unconditionally skip `aws-sdk` and all dependencies
        # (Because already installed in target Lambda)
        - "aws-sdk"
      
      # Specify import misses that are anticipated and allowed.
      allowMissing: 
        # ...

      dynamic:
        # Explicitly handle all dynamic import misses by ignoring or adding
        # additional files to trace.
        #
        # See: https://npm.im/serverless-jetpack#handling-dynamic-import-misses
        resolutions:
          # ...

I do realize this is a bit of config to bite off from the relative simplicity of the webpack plugin, but once you've got the hang of things we'll hope you like it -- we've had pretty good responses from folks switching from the webpack plugin to Jetpack.

And if you're encountering config issues getting started with Jetpack, feel free to drop them in here on this issue or create a public, minimal repo to replicate the structural problem you're trying to solve and I can jump in.

Cheers and good luck!

[vamcs]

@ryan-roemer wow, thanks a lot for such a quick and comprehensive answer! I think I only missed tracing when I first set it up today but I'll continue tomorrow and report back before closing this issue. 🙂

[vamcs]

I've managed to get it to work and the generate bundles were really small, like ~65KB each (the lambdas are very simple as of now though - they just basically reach for DynamoDB). But I'll can show how I set up the environment in case anyone else needs some help with it too:

My Typescript setup with Babel, ESLint, Prettier and Serverless Jetpack:

First of all, my project structure as of now is:

.
├── .eslintignore
├── .eslintrc.json
├── .gitignore
├── .prettierrc
├── babel.config.json
├── dist
├── package.json
├── serverless.yml
├── services
│   ├── company
│   └── user
├── shared
│   ├── helpers
│   └── models
├── tsconfig.json
└── yarn.lock

My initial plan was to also have a serverless.yml and package.json for each service and deploy them separately but I'll skip that for now and most likely change to that setup in the future. So I'm using Babel for transpiling TS to JS (this was needed because I use alias paths and it was easier to use a Babel plugin to fix that), Prettier, ESLint and I do some type checking using tsc during my build.

The relevant dev dependencies I use for the setup are:

"devDependencies": {
  "@babel/cli": "^7.10.5",
  "@babel/core": "^7.11.1",
  "@babel/preset-env": "^7.11.0",
  "@babel/preset-typescript": "^7.10.4",
  "@typescript-eslint/eslint-plugin": "^3.8.0",
  "@typescript-eslint/parser": "^3.8.0",
  "babel-jest": "^26.2.2",
  "babel-plugin-const-enum": "^1.0.1",
  "babel-plugin-module-resolver": "^4.0.0",
  "eslint": "^7.6.0",
  "eslint-config-airbnb-base": "^14.2.0",
  "eslint-config-prettier": "^6.11.0",
  "eslint-import-resolver-alias": "^1.1.2",
  "eslint-import-resolver-typescript": "^2.2.0",
  "eslint-plugin-import": "^2.22.0",
  "eslint-plugin-jest": "^23.20.0",
  "eslint-plugin-module-resolver": "^0.17.0",
  "eslint-plugin-prettier": "^3.1.4",
  "jest": "^26.2.2",
  "prettier": "^2.0.5",
  "serverless-jetpack": "^0.10.7",
  "serverless-offline": "^6.5.0",
  "serverless-prune-plugin": "^1.4.3",
  "serverless-scriptable-plugin": "^1.0.5",
  "typescript": "^3.2.4"
}

The Babel config:

{
  "plugins": [
    "const-enum",
    [
      "module-resolver",
      {
        "root": ["."],
        "alias": {
          "@my-project/services": "./services",
          "@my-project/shared": "./shared"
        }
      }
    ]
  ],
  "presets": [
    [
      "@babel/preset-env",
      { "modules": "commonjs", "targets": { "node": "12" } }
    ],
    ["@babel/preset-typescript"]
  ],
  "ignore": ["node_modules", "dist"]
}

The ESLint config

{
  "extends": [
    "airbnb-base",
    "plugin:jest/all",
    "plugin:import/errors",
    "plugin:import/warnings",
    "plugin:import/typescript",
    "plugin:@typescript-eslint/recommended",
    "prettier"
  ],
  "plugins": ["jest", "@typescript-eslint", "eslint-plugin-import"],
  "root": true,
  "globals": {},
  "rules": {
    "import/no-unresolved": [2, { "commonjs": true, "amd": true }],
    "import/prefer-default-export": "off",
    "@typescript-eslint/explicit-function-return-type": "off",
    "@typescript-eslint/explicit-module-boundary-types": "off",
    "camelcase": "off",
    "no-console": "off",
    "import/extensions": "off"
  },
  "parser": "@typescript-eslint/parser",
  "parserOptions": {
    "ecmaVersion": 2020,
    "sourceType": "module"
  },
  "env": {
    "jest/globals": true
  },
  "overrides": [],
  "settings": {
    "import/resolver": {
      "typescript": { "alwaysTryTypes": true },
      "node": {
        "extensions": [".js", ".ts"],
        "paths": ["node_modules/", "node_modules/@types"]
      },
      "alias": {
        "map": [
          ["@my-project/services/", "./services"],
          ["@my-project/shared/", "./shared"]
        ]
      }
    },
    "jest": {
      "version": 26
    }
  }
}

The relevant parts of my serverless.yml:

custom:
 ...
 jetpack:
    concurrency: 4
    collapsed:
      bail: true
    preInclude:
      - "!**"
    trace:
      dynamic:
        bail: true

package:
  individually: true
  include:
    - "!yarn.lock"
    - "!node_modules/aws-sdk/**"

plugins:
  - serverless-scriptable-plugin
  - serverless-offline
  - serverless-prune-plugin
  - serverless-jetpack

My tsconfig.json:

{
  "compilerOptions": {
    "strict": true,
    "preserveConstEnums": true,
    "module": "commonjs",
    "lib": ["esnext"],
    "forceConsistentCasingInFileNames": true,
    "removeComments": true,
    "moduleResolution": "node",
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "target": "es2017",
    "outDir": "dist",
    "inlineSources": true,
    "inlineSourceMap": true,
    "sourceRoot": "./services",
    "typeRoots": ["node_modules/@types"],
    "esModuleInterop": true,
    "baseUrl": "./",
    "paths": {
      "@my-service/services/*": ["./services/*"],
      "@my-service/shared/*": ["./shared/*"]
    }
  },
  "include": ["./**/*.ts"],
  "exclude": [
    "node_modules/**/*",
    ".serverless/**/*",
    ".webpack/**/*",
    "_warmup/**/*",
    ".vscode/**/*"
  ]
}

My build scripts in package.json. I first clean up the buid directory, lint my code, do some type checking with tsc and then I build everything with Babel. The Babel build is also saved in dist, same as the outDir for tsconfig.

"scripts": {
  "test": "NODE_ENV=test jest --ci --verbose",
  "lint": "eslint services shared --ext .ts",
  "build:types": "yarn run tsc --noEmit",
  "build:js": "yarn run babel . --out-dir dist --extensions \".ts\" --source-maps inline",
  "build": "rm -rf dist && yarn lint && yarn build:types && yarn build:js"
},

There you go! Thanks a lot for this library and hope this is useful to someone else 🤠