flowrunner package

Subpackages

Submodules

flowrunner.cli module

Module for cli commands Usage: python -m flowrunner [OPTIONS] COMMAND [ARGS]…

Welcome to flowRunner! 🚀

flowRunner is a lightweight package to organize and represent Data Engineering/Science workflows. Its designed to be integrated with any pre- existing framework like pandas or PySpark

Main FeaturesEasy BaseFlow to use to build Flows off ofSimple decorators to convert methods to Flow methodsCommand Line Interface for running Flows

Options:

--help: Show this message and exit.

Commands:

run Command to run a Flow show Command to show the order of iteration of a Flow validate Command to validate a Flow directory Command to visualize a directory

Module contents

Welcome to FlowRunner!

FlowRunner is a lightweight package to organize and represent Data Engineering/Science workflows. Its designed to be integrated with any pre-existing framework like pandas or PySpark

Main Features - Easy BaseFlow to use to build Flows off of - Simple decorators to convert methods to Flow methods - Command Line Interface for running Flows

```python # example.py from flowrunner import BaseFlow, step, start, end

class ExampleFlow(BaseFlow):

@start @step(next=[‘method2’, ‘method3’]) def method1(self):

self.a = 1

@step(next=[‘method4’]) def method2(self):

self.a += 1

@step(next=[‘method4’]) def method3(self):

self.a += 2

@end @step def method4(self):

self.a += 3 print(self.a)

```

You can run the flow with the following command `sh $ python -m flowrunner example.py 7 `