In the previous part of this article we've finished our Discord bot by adding some basic database support and a slash command to add new cat facts to our database. It can now also select a random fact from the ones available and display it after someone triggers it with a !catfact. It works, but does it really?
The problem is that for the two whole articles, we've been testing our bot by simply running it and interacting with it on an actual live Discord server. This is obviously a very poor way to do any testing, so let's improve our project by writing some automated tests that can be easily executed from the console with the mix test command. It's very similar to what we are used to doing in our Rails projects, though it will require a bit more manual setup due to our bot not using any frameworks.
As always, you can find the code examples in the github repository.
Elixir for Rubyists - Cat Facts in Discord (Part 3)
In the previous part of this article we've finished our Discord bot by adding some basic database support and a slash command to add new cat facts to our database. It can now also select a random fact from the ones available and display it after someone triggers it with a !catfact. It works, but does it really?
The problem is that for the two whole articles, we've been testing our bot by simply running it and interacting with it on an actual live Discord server. This is obviously a very poor way to do any testing, so let's improve our project by writing some automated tests that can be easily executed from the console with the mix test command. It's very similar to what we are used to doing in our Rails projects, though it will require a bit more manual setup due to our bot not using any frameworks.
As always, you can find the code examples in the github repository.
Setting up the testing framework
Any Elixir project already comes with the testing framework ExUnit, but we could also use support from mocks and factories, and we can also install a code coverage tool, so that we can see what still needs to be tested. Additionally, since we're using a database to save our facts, we need to separate it from the development or production databases. First, we need some extra packages. ExMachina is the Elixir's equivalent of FactoryBot, ExCoveralls and the last one, Patch is a mocking library.
We'll add them in the mix.exs
file just like the last time by adding the necessary entries to the defp deps
:
{:ex_machina, "~> 2.7.0", only: :test},
{:excoveralls, "~> 0.10", only: :test},
{:patch, "~> 0.13", only: :test}
As you can see, they all are marked with only: :test
, which as expected will skip compiling these packages unless we're using the test environment. We also need a little bit of extra configuration for the coverage package to work. This is also done in the same file. We're also adding an extra directory -
def project do
[
app: :cat_facts,
version: "0.1.0",
elixir: "~> 1.15",
elixirc_paths: elixirc_paths(Mix.env()),
start_permanent: Mix.env() == :prod,
deps: deps(),
test_coverage: [tool: ExCoveralls],
preferred_cli_env: [
coveralls: :test,
"coveralls.detail": :test,
"coveralls.post": :test,
"coveralls.html": :test
]
]
end
defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]
In the new
defmodule CatFacts.DataCase do
use ExUnit.CaseTemplate
using do
quote do
alias CatFacts.Repo
import Ecto
import Ecto.Changeset
import Ecto.Query
import CatFacts.DataCase
end
end
setup tags do
CatFacts.DataCase.setup_sandbox(tags)
:ok
end
def setup_sandbox(tags) do
pid = Ecto.Adapters.SQL.Sandbox.start_owner!(CatFacts.Repo, shared: not tags[:async])
on_exit(fn -> Ecto.Adapters.SQL.Sandbox.stop_owner(pid) end)
end
end
Before we start running any tests, we also need a bit of extra configuration for our databases - we don't want to test anything on our development database after all, so we need to introduce a separate database for testing. For that, we need to split up our development and test configs. Back in the config
directory, make a new file test.exs
for that:
import Config
config :cat_facts, CatFacts.Repo,
username: "user",
password: "password",
hostname: "localhost",
database: "cat_facts_test#{System.get_env("MIX_TEST_PARTITION")}",
pool: Ecto.Adapters.SQL.Sandbox,
pool_size: 10
config :logger, level: :warning
And similarly, we can move the database config for the development environment from config.exs
to a new dev.exs
file:
import Config
config :cat_facts, CatFacts.Repo,
database: "cat_facts_dev",
username: "user",
password: "password",
hostname: "localhost",
stacktrace: true,
show_sensitive_data_on_connection_error: true,
pool_size: 10
After this, the main config.exs
file should look like this, which will define the common values and load the appropriate database configuration for the right environment.
import Config
config :cat_facts, ecto_repos: [CatFacts.Repo]
config :nostrum,
token: System.get_env("CATFACT_BOT_TOKEN"),
gateway_intents: :all
config :logger, :console,
metadata: [:shard, :guild, :channel]
# Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above.
import_config "#{config_env()}.exs"
Last but not least, we need a test file. Make a folder in the test
directory named cat_facts and in that folder a new file, facts_test.exs
. Just like in Rails and RSpec, the tests need to have that specific _test.exs
naming scheme and extension in order to be picked up by the testing suite.
defmodule CatFacts.FactsTest do
use CatFacts.DataCase
doctest CatFacts.Facts, import: true
end
This test file will be using our DataCase
we introduced earlier - so that the data doesn't linger between different tests - and we'll be using doctests to test our application's Facts
context.
Testing database interactions
Now we can run mix deps.get
from the console and then mix coveralls.html
. We should see some output from the coveralls package in the console and in the cover directory we should get an excoveralls.html
file which shows all our project's files that get compiled and how much of the code in them is covered by our tests.
As you can see, pretty much nothing is tested at this point, except for the fact that our application does start correctly. Since we've already declared that we'll be using doctests in our FactsTest
test file above, let's actually write some. To do so, we need to go back to our lib/cat_facts/facts.ex
file - that's where doctests go, rather than being separated in the test file itself.
@doc """
Creates a fact for the specified Discord guild with the supplied content.
## Examples
iex> {:ok, %CatFacts.Facts.Fact{} = result} = create_guild_fact(12345, "my cool fact")
iex> result.guild_id == 12345
true
iex> {:ok, %CatFacts.Facts.Fact{} = result} =
...> create_guild_fact(12345, "my cool fact")
iex> result.content == "my cool fact"
true
"""
def create_guild_fact(guild_id, content) do
%Fact{}
|> Fact.changeset(%{guild_id: guild_id, content: content})
|> Repo.insert()
end
What we've written here is a doctest - a documentation block for a function that not only describes what the function does, but also provides one or more code examples of the function's usage. Each of those examples will also be treated as a separate test case. Here we're declaring two tests - they will add a fact into the database and then check that the returned record does have the parameters we've passed to the function.
A doctest consists of two parts, just like a regular test does - we can see a few lines of setup, which are marked by the iex>
(and ...> in case of multiline commands) prefixes. The result of those setup commands will be compared to the line without a prefix - in those cases, to true
, since the last setup command is a comparison. We can add a similar test for the second function in our Facts
context:
@doc """
Returns a randomly chosen fact for the specified guild ID or nil if there aren't any facts present.
## Examples
iex> get_random_guild_fact(99887)
nil
iex> create_guild_fact(123, "coolest fact")
iex> create_guild_fact(543, "this one's uncool")
iex> result = get_random_guild_fact(123)
iex> result.content == "coolest fact"
true
"""
def get_random_guild_fact(guild_id) do
query =
from f in Fact,
where: f.guild_id == ^guild_id,
order_by: fragment("RANDOM()"),
limit: 1
Repo.one(query)
end
Now with the two doctests and four examples in them, we can run the mix coveralls.html tool again and see what else needs to be covered with tests.
If we open the generated html file, we can see that it found only 3 relevant lines of code in our Facts
context and that our tests checked all of them. Looking at the right side of the report to see what files are left untested, we can see that we should move to testing the bot_consumer.ex
file, which has most of the code in this project.
Writing regular tests
We'll put the new tests in a new file, which should be saved as test/cat_facts/bot_consumer_test.exs
. At first, we can pick up the easiest test we have to write in the BotConsumer
module - dealing with unhandled messages, to which our function should return a :noop
atom.
defmodule CatFacts.BotConsumerTest do
use CatFacts.DataCase
use Patch
alias CatFacts.BotConsumer
alias Nostrum.Struct.{WSState, Event, Guild, Interaction, Message}
describe "on unhandled events" do
setup do
reaction_event = {:MESSAGE_REACTION_ADD, %Event.MessageReactionAdd{}, %WSState{}}
{:ok, event: reaction_event}
end
test "handler returns noop", %{event: event} do
assert BotConsumer.handle_event(event) == :noop
end
end
end
Let's walk through this code before we move on to more advanced tests.
use CatFacts.DataCase
use Patch
alias CatFacts.BotConsumer
alias Nostrum.Struct.{WSState, Event, Guild, Interaction, Message}
First we got a block of - mostly unused for now - aliases and uses. Aliases are used for code readability, just like in regular code, but the two use
macros are more interesting. First we make sure that we're using the DataCase
we've introduced earlier into test/support/data_case.exs
. This will make sure all the tests are separated from each other. The other use
macro introduces the Patch
package we added at the start of this article. It will allow us to test our code while not actually connecting to any actual Discord servers.
describe "on unhandled events" do
setup do
reaction_event = {:MESSAGE_REACTION_ADD, %Event.MessageReactionAdd{}, %WSState{}}
{:ok, event: reaction_event}
end
test "handler returns noop", %{event: event} do
assert BotConsumer.handle_event(event) == :noop
end
end
Next we have our actual test. While generally similar to how we write tests in Rails, there are some differences. In Rails, we'd normally use RSpec the setup would look similar to this:
let(:event) { {:MESSAGE_REACTION_ADD, %Event.MessageReactionAdd{}, %WSState{}} }
And the event
variable declared in this way would be instantly available in the test itself. Elixir is a much more explicit language though, so the test setup is also a little more explicit.
setup do
reaction_event = {:MESSAGE_REACTION_ADD, %Event.MessageReactionAdd{}, %WSState{}}
{:ok, event: reaction_event}
end
First we declare our event inside the setup
block. In order for it to be available inside the test, this block needs to return a tuple starting with an :ok
atom and followed with whatever variables we want to have available in the tests.
test "handler returns noop", %{event: event} do
This tuple is then passed as a context to the test itself, where we can use pattern matching to extract the variables we want.
assert BotConsumer.handle_event(event) == :noop
The test itself on the other hand is relatively straightforward - there are the expected assertions and refutals available to us, so just like the line above, we're asserting that our function will return the expected result. Next we can move on to other types of events to be handled.
describe "on INTERACTION event" do
setup do
interaction = %Interaction{
guild_id: 12345,
data: %{
name: "fact",
options: [
%{name: "content", value: "very interesting fact"}
]
}
}
interaction_event = {:INTERACTION_CREATE, interaction, %WSState{}}
expected_response = %{
data: %{
flags: 64,
content: "Your cat fact has been added to this server's list of cat facts."
},
type: 4
}
{:ok, event: interaction_event, response: expected_response}
end
test "handler creates a response", %{event: {_, interaction, _} = event, response: response} do
patch(Nostrum.Api, :create_interaction_response, nil)
BotConsumer.handle_event(event)
assert_called Nostrum.Api.create_interaction_response(^interaction, ^response)
end
end
Once again most of this new code is taken by the setup
macro. We're creating an interaction_event
tuple just like in the previous example, but we also prepare a map of a response that we expect to be used. We can pass it to the test context.
test "handler creates a response", %{event: {_, interaction, _} = event, response: response} do
Since we also need our initial interaction to be used in the assertion, we can use pattern matching to extract it from the passed event, rather than passing it separately.
patch(Nostrum.Api, :create_interaction_response, nil)
Next we'll patch the Nostrum library, so that the create_interaction_response
function responds with a nil instead of whatever the function's code originally was. This way it doesn't call the real Discord API anymore. Now we can easily focus on testing only our own module, and not the external services or libraries - after all, we can safely assume the creators of those test them on their own, so it's not our responsibility.
BotConsumer.handle_event(event)
assert_called Nostrum.Api.create_interaction_response(^interaction, ^response)
Finally, we call our handle_event
function and check with the assert_called
if our function called the Nostrum API in the expected way. In this particular assertion, we need to remember to use variable pinning if we want to check for specific values, otherwise it would just compare types and number of arguments.
Similarly, we can write a test for the READY
event - the code for that test is pretty much the same as here, so you can check it out in the github repository.
Using factories for database records
Let's move on to our final part of the BotConsumer
module, which is testing the response for different messages. Since that function should behave differently depending on the existence of records in the database, this is a good opportunity to use a Factory. Before we do so, we need to make some extra changes to our test setup.
In the mix.exs
file, we need to add the test/factories
directory to be compiled in the test environment, similarly to what we did for test/support
. This will allow us to have a nice and logical directory structure.
defp elixirc_paths(:test), do: ["lib", "test/support", "test/factories"]
Next we need to take a look at the test/test_helper.exs
file and start ExMachina there. ExMachina is Elixir's equivalent of FactoryBot - a library that deals with factories.
ExUnit.start()
{:ok, _} = Application.ensure_all_started(:ex_machina)
We also need a place to list and manage our factories, and that will go into the test/support/factory.ex
file. Here we can list all our factories and make sure all of them are imported into our tests. We could place all our factories in this file just as functions in this Factory
module, but I think Rails has a good idea here to keep those in separate files, and we can do that easily enough.
defmodule CatFacts.Factory do
use ExMachina.Ecto, repo: CatFacts.Repo
use CatFacts.FactFactory
end
And finally, the factory file itself - test/factories/fact_factory.ex
. The extra lines of defmacro
allow us to handle the use
macro in the code above.
defmodule CatFacts.FactFactory do
alias CatFacts.Facts.Fact
defmacro __using__(_opts) do
quote do
def fact_factory do
%Fact{
guild_id: 123,
content: "a normal fact"
}
end
end
end
end
With our setup finally complete, we can write the last tests that we need to have full code coverage. Since most of this code is very similar to other tests in this article, we can focus only on the one that uses our new factory. You can always check out the full test in the github repository.
describe "on MESSAGE event" do
# ...
test "handles the !catfact command with existing fact", %{ok_event: event} do
insert(:fact, guild_id: 1234, content: "cool fact")
patch(Nostrum.Api, :create_message, nil)
BotConsumer.handle_event(event)
assert_called Nostrum.Api.create_message(456, "cool fact")
end
end
The factory works exactly like we are used to with FactoryBot - we can insert()
a :fact
, and specify (or omit any of them, in which they would fall back to the defaults from the FactFactory
module) the attributes of the database record we're inserting. Similarly to FactoryBot, we could also use the build
function instead and have a record that's not yet saved - to check for validations for example.
Wrapping up
With this we can finally consider our little project complete. In the first two articles we have finished the code and now all our code is able to be automatically tested without a need to manually go on Discord. Since we have learned about doctests, in a next project we should make sure to write them together with the code, and we also have learned how to mock external libraries, so that our tests can be focused on testing our own code.
Obviously our Discord bot turned out to be pretty simple, so the tests aren't very complicated either, but the basic principles behind them will stay the same in pretty much any sized project.
On-demand webinar: Twilio implementation in GreenWay
Check how GreenWay improved customer engagement by implementing Twilio Studio with the assistance of our team.
Latest blog posts
Ready to talk about your project?
Tell us more
Fill out a quick form describing your needs. You can always add details later on and we’ll reply within a day!
Strategic Planning
We go through recommended tools, technologies and frameworks that best fit the challenges you face.
Workshop Kickoff
Once we arrange the formalities, you can meet your Polcode team members and we’ll begin developing your next project.