Migrations¶
You will almost certainly need to be using a database migration tool to make sure you manage your incremental database changes properly.
Edgy being on the top of SQLAlchemy core means that we can leverage that within the internal migration tool.
Edgy provides an internal migration tool that makes your life way easier when it comes to manage models and corresponding migrations.
Heavily inspired by the way Flask-Migration approached the problem, Edgy took it to the next level and makes it framework agnostic, which means you can use it anywhere.
Important¶
Before reading this section, you should get familiar with the ways Edgy handles the discovery of the applications.
The following examples and explanations will be using the --app and environment variables approach but the auto discovery is equally valid and works in the same way.
Structure being used for this document¶
For the sake of this document examples and explanations we will be using the following structure to make visually clear.
.
└── README.md
└── .gitignore
└── myproject
├── __init__.py
├── apps
│ ├── __init__.py
│ └── accounts
│ ├── __init__.py
│ ├── tests.py
│ └── v1
│ ├── __init__.py
│ ├── schemas.py
│ ├── urls.py
│ └── views.py
├── configs
│ ├── __init__.py
│ ├── development
│ │ ├── __init__.py
│ │ └── settings.py
│ ├── settings.py
│ └── testing
│ ├── __init__.py
│ └── settings.py
├── main.py
├── serve.py
├── utils.py
├── tests
│ ├── __init__.py
│ └── test_app.py
└── urls.py
Migration¶
This is the object that Edgy requires to make sure you can manage the migrations in a consistent, clean and simple manner. Much like Django migrations type of feeling.
This Migration class is not depending of any framework specifically, in fact, Edgy makes sure
when this object is created, it will plug it into any framework you desire.
This makes Edgy unique and extremely flexible to be used within any of the Frameworks out there, such as Esmerald, Starlette, FastAPI, Sanic... You choose.
from edgy import Instance, monkay
monkay.set_instance(Instance(registry=registry, app=None))
Parameters¶
The parameters availabe when using instantiating a Instance object are the following:
- registry - The registry being used for your models. The registry must be an instance
of
edgy.Registryor anAssertationErroris raised. - app - Optionally an application instance.
Settings¶
The following settings are available in the main settings object:
- multi_schema (bool / regexstring / regexpattern) - Activate multi schema migrations (Default: False).
- ignore_schema_pattern (None / regexstring / regexpattern) - When using multi schema migrations, ignore following regex pattern (Default "information_schema")
- alembic_ctx_kwargs (dict) - Extra arguments for alembic. By default:
{ "compare_type": True, "render_as_batch": True, } - migration_directory (str / PathLike) - Migrations directory. Absolute or relative. By default: "migrations".
How to use it¶
Using the Instance class is very simple in terms of requirements. In the tips and tricks you can see some examples in terms of using the LRU cache technique. If you haven't seen it, it is recommended you to have a look.
For this examples, we will be using the same approach.
Assuming you have a utils.py where you place your information about the database and
registry.
Something like this:
from functools import lru_cache
from edgy import Database, Registry
@lru_cache()
def get_db_connection():
database = Database("postgresql+asyncpg://user:pass@localhost:5432/my_database")
return database, Registry(database=database)
This will make sure we don't create objects. Nice technique and quite practical.
Now that we have our details about the database and registry, it is time to use the Migration object in the application.
Using Esmerald¶
#!/usr/bin/env python
import os
import sys
from pathlib import Path
from my_project.utils import get_db_connection
from edgy import Instance, monkay
from esmerald import Esmerald, Include
def build_path():
"""
Builds the path of the project and project root.
"""
Path(__file__).resolve().parent.parent
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
if SITE_ROOT not in sys.path:
sys.path.append(SITE_ROOT)
sys.path.append(os.path.join(SITE_ROOT, "apps"))
def get_application():
"""
This is optional. The function is only used for organisation purposes.
"""
build_path()
registry = get_db_connection()
app = registry.asgi(
Esmerald(
routes=[Include(namespace="my_project.urls")],
)
)
monkay.set_instance(Instance(registry=registry, app=app))
return app
app = get_application()
Using FastAPI¶
As mentioned before, Edgy is framework agnostic so you can also use it in your FastAPI application.
#!/usr/bin/env python
import os
import sys
from pathlib import Path
from fastapi import FastAPI
from my_project.utils import get_db_connection
from edgy import Instance, monkay
def build_path():
"""
Builds the path of the project and project root.
"""
Path(__file__).resolve().parent.parent
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
if SITE_ROOT not in sys.path:
sys.path.append(SITE_ROOT)
sys.path.append(os.path.join(SITE_ROOT, "apps"))
def get_application():
"""
This is optional. The function is only used for organisation purposes.
"""
build_path()
registry = get_db_connection()
app = registry.asgi(FastAPI(__name__))
monkay.set_instance(Instance(registry=registry, app=app))
return app
app = get_application()
Using Starlette¶
The same goes for Starlette.
#!/usr/bin/env python
import os
import sys
from pathlib import Path
from lilya.apps import Lilya
from my_project.utils import get_db_connection
from edgy import monkay, Instance
def build_path():
"""
Builds the path of the project and project root.
"""
Path(__file__).resolve().parent.parent
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
if SITE_ROOT not in sys.path:
sys.path.append(SITE_ROOT)
sys.path.append(os.path.join(SITE_ROOT, "apps"))
def get_application():
"""
This is optional. The function is only used for organisation purposes.
"""
build_path()
registry = get_db_connection()
app = registry.asgi(Lilya(__name__))
monkay.set_instance(Instance(app=app, registry=registry))
return app
app = get_application()
Using other frameworks¶
I believe you got the idea with the examples above, It was not specified any special framework unique-like parameter that demanded special attention, just the application itself.
This means you can plug something else like Quart, Ella or even Sanic... Your pick.
Example¶
Let us assume we have an application with the following structure.
.
└── README.md
└── .gitignore
└── myproject
├── __init__.py
├── apps
│ ├── __init__.py
│ └── accounts
│ ├── __init__.py
│ ├── tests.py
│ ├── models.py
│ └── v1
│ ├── __init__.py
│ ├── schemas.py
│ ├── urls.py
│ └── views.py
├── configs
│ ├── __init__.py
│ ├── development
│ │ ├── __init__.py
│ │ └── settings.py
│ ├── settings.py
│ └── testing
│ ├── __init__.py
│ └── settings.py
├── main.py
├── serve.py
├── utils.py
├── tests
│ ├── __init__.py
│ └── test_app.py
└── urls.py
As you can see, it is quite structured but let us focus specifically on accounts/models.py.
There is where your models for the accounts application will be placed. Something like this:
from datetime import datetime
from my_project.utils import get_db_connection
import edgy
_, registry = get_db_connection()
class User(edgy.Model):
"""
Base model for a user
"""
first_name: str = edgy.CharField(max_length=150)
last_name: str = edgy.CharField(max_length=150)
username: str = edgy.CharField(max_length=150, unique=True)
email: str = edgy.EmailField(max_length=120, unique=True)
password: str = edgy.CharField(max_length=128)
last_login: datetime = edgy.DateTimeField(null=True)
is_active: bool = edgy.BooleanField(default=True)
is_staff: bool = edgy.BooleanField(default=False)
is_superuser: bool = edgy.BooleanField(default=False)
class Meta:
registry = registry
Now we want to tell the Instance object to make sure it knows about this.
#!/usr/bin/env python
import os
import sys
from pathlib import Path
from my_project.utils import get_db_connection
from edgy import Instance, monkay
from esmerald import Esmerald, Include
def build_path():
"""
Builds the path of the project and project root.
"""
Path(__file__).resolve().parent.parent
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
if SITE_ROOT not in sys.path:
sys.path.append(SITE_ROOT)
sys.path.append(os.path.join(SITE_ROOT, "apps"))
def get_application():
"""
This is optional. The function is only used for organisation purposes.
"""
build_path()
registry = get_db_connection()
app = registry.asgi(
Esmerald(
routes=[Include(namespace="my_project.urls")],
)
)
monkay.set_instance(
Instance(
app=app,
registry=registry,
)
)
return app
app = get_application()
Generating and working with migrations¶
Now this is the juicy part, right? Yes but before jumping right into this, please make sure you read properly the migration section and you have everything in place.
It is recommended that you follow the environment variables suggestions.
This will depend heavily on this and everything works around the registry.
Edgy has the internal client that manages and handles the migration process for you in a clean
fashion and it called edgy.
Remember the initial structure at the top of this document? No worries, let us have a look again.
.
└── README.md
└── .gitignore
└── myproject
├── __init__.py
├── apps
│ ├── __init__.py
│ └── accounts
│ ├── __init__.py
│ ├── tests.py
│ └── v1
│ ├── __init__.py
│ ├── schemas.py
│ ├── urls.py
│ └── views.py
├── configs
│ ├── __init__.py
│ ├── development
│ │ ├── __init__.py
│ │ └── settings.py
│ ├── settings.py
│ └── testing
│ ├── __init__.py
│ └── settings.py
├── main.py
├── serve.py
├── utils.py
├── tests
│ ├── __init__.py
│ └── test_app.py
└── urls.py
This structure is important as it will make it easier to explain where you should start with migrations.
Note
Using the above structure helps for visual purposes but by the end of this document, you don't need to follow this way, you can do whatever you want.
Danger
You can generate the migrations anywhere in your codebase but you need to be careful about the paths and all of the internal dependencies. It is recommended to have them at the root of your project, but again, up to you.
Assuming you have your application inside that my_project/main.py the next steps will follow
that same principle.
Environment variables¶
When generating migrations, Edgy expects at least one environment variable to be present.
- EDGY_DATABASE_URL - The database url for your database.
The reason for this is because Edgy is agnostic to any framework and this way it makes it easier
to work with the migrations.
Also, gives a clean design for the time where it is needed to go to production as the procedure is very likely to be done using environment variables.
This variable must be present. So to save time you can simply do:
$ export EDGY_DATABASE_URL=postgresql+asyncpg://user:pass@localhost:5432/my_database
Or whatever connection string you are using.
Initialize the migrations folder¶
It is now time to generate the migrations folder. As mentioned before in the
environment variables section, Edgy does need to have the
EDGY_DATABASE_URL to generate the migrations folder. So, without further ado let us generate
our migrations.
edgy --app myproject.main init
What is happenening here? Well, edgy is always expecting an --app parameter to be
provided.
This --app is the location of your application in module_app format and this is because of
the fact of being framework agnostic.
Edgy needs the module automatically setting the instance (see Connections) to know the registry which shall be used as well as the application object.
Remember when it was mentioned that is important the location where you generate the migrations
folder? Well, this is why, because when you do my_project.main you are telling that
your application is inside the myproject/main/app.py and your migration folder should be placed
where the command was executed.
In other words, the place you execute the init command it will be where the migrations will be
placed.
Let us see how our structrure now looks like after generating the migrations.
.
└── README.md
└── .gitignore
├── migrations
│ ├── alembic.ini
│ ├── env.py
│ ├── README
│ ├── script.py.mako
│ └── versions
└── myproject
├── __init__.py
├── apps
│ ├── __init__.py
│ └── accounts
│ ├── __init__.py
│ ├── tests.py
│ └── v1
│ ├── __init__.py
│ ├── schemas.py
│ ├── urls.py
│ └── views.py
├── configs
│ ├── __init__.py
│ ├── development
│ │ ├── __init__.py
│ │ └── settings.py
│ ├── settings.py
│ └── testing
│ ├── __init__.py
│ └── settings.py
├── main.py
├── serve.py
├── utils.py
├── tests
│ ├── __init__.py
│ └── test_app.py
└── urls.py
Pretty great so far! Well done 🎉🎉
You have now generated your migrations folder and came with gifts.
A lot of files were generated automatically for you and they are specially tailored for the needs and complexity of Edgy.
Do you remember when it was mentioned in the environment variables that
edgy is expecting the EDGY_DATABASE_URL to be available?
Well, this is another reason, inside the generated migrations/env.py the get_engine_url() is
also expecting that value.
# Code above
def get_engine_url():
return os.environ.get("EDGY_DATABASE_URL")
# Code below
Warning
You do not need to use this environment variable. This is the default provided by Edgy.
You can change the value to whatever you want/need but be careful when doing it as it might
cause Edgy not to work properly with migrations if this value is not updated properly.
Generate the first migrations¶
Now it is time to generate your first migration.
Assumming we want to place the models for an accounts application inside a models.py.
Let us define our User model.
from datetime import datetime
from my_project.utils import get_db_connection
import edgy
_, registry = get_db_connection()
class User(edgy.Model):
"""
Base model for a user
"""
first_name: str = edgy.CharField(max_length=150)
last_name: str = edgy.CharField(max_length=150)
username: str = edgy.CharField(max_length=150, unique=True)
email: str = edgy.EmailField(max_length=120, unique=True)
password: str = edgy.CharField(max_length=128)
last_login: datetime = edgy.DateTimeField(null=True)
is_active: bool = edgy.BooleanField(default=True)
is_staff: bool = edgy.BooleanField(default=False)
is_superuser: bool = edgy.BooleanField(default=False)
class Meta:
registry = registry
Now we need to make sure the models are accessible in the application for discovery. Since
this example is based on Esmerald scaffold, simply add your User model into the
my_project/apps/accounts/__init__.py.
from .models import User
Note
Since Edgy is agnostic to any framework, there aren't automatic mechanisms that detects
Edgy models in the same fashion that Django does with the INSTALLED_APPS. So this is
one way of exposing your models in the application.
There are many ways of exposing your models of course, so feel free to use any approach you want.
Now it is time to generate the migration.
$ edgy --app my_project.main makemigrations
Yes, it is this simple 😁
Your new migration should now be inside migrations/versions/. Something like this:
.
└── README.md
└── .gitignore
├── migrations
│ ├── alembic.ini
│ ├── env.py
│ ├── README
│ ├── script.py.mako
│ └── versions
│ └── d3725dd11eef_.py
└── myproject
...
Or you can attach a message your migration that will then added to the file name as well.
$ edgy --app my_project.main makemigrations -m "Initial migrations"
.
└── README.md
└── .gitignore
├── migrations
│ ├── alembic.ini
│ ├── env.py
│ ├── README
│ ├── script.py.mako
│ └── versions
│ └── d3725dd11eef_initial_migrations.py
└── myproject
...
Migrate your database¶
Now comes the easiest part where you need to apply the migrations.
Simply run:
$ edgy --app my_project.main:app migrate
And that is about it 🎉🎉
You have managed to create the migrations, generate the files and migrate them in some simple steps.
Change the models and generate the migrations¶
Well, it is not rocket science here. You can change your models as you please like you would do for any other ORM and when you are happy run the migrations and apply them again by running:
Generate new migrations
$ edgy --app my_project.main makemigrations
Apply them to your database
$ edgy --app my_project.main migrate
More migration commands¶
There are of course more available commands to you to be used which they can also be accessed
via --help command.
Edgy admin¶
To access the available options of edgy:
$ edgy --help
This will list all the commands available within edgy.
What if you need to also know the available options available to each command?
Let us imagine you want to see the available options for the merge
$ edgy merge --help
You should see something like this:
Usage: edgy merge [OPTIONS] [REVISIONS]...
Merge two revisions together, creating a new revision file
Options:
--rev-id TEXT Specify a hardcoded revision id instead of generating
one
--branch-label TEXT Specify a branch label to apply to the new revision
-m, --message TEXT Merge revision message
-d, --directory TEXT Migration script directory (default is "migrations")
--help Show this message and exit.
This is applied to any other available command via edgy.
References¶
Since Edgy has a very friendly and familiar interface to interact with so does the
edgy.
Edgy migrations as mentioned before uses Alembic and therefore the commands are exactly the same as the ones for alembic except two, which are masked with different more intuitive names.
- makemigrations - Is calling the Alembic
migrate. - migrate - Is calling the Alembic
upgrade.
Since the alembic names for those two specific operations is not that intuitive, Edgy masks them into a more friendly and intuitive way.
For those familiar with Django, the names came from those same operations.
Very important¶
Check the environment variables for more details and making sure you follow the right steps.