# Understanding distro's config mechanism

Ubuntu 20.04 latest ami seems to start using EC2 instance connect by default. And if you want to muck around with it, you can start from `/lib/systemd/system/ssh` and `/lib/systemd/system/ssh.service.d/ec2-instance-connect.conf`, not in `/etc/ssh/sshd_config`.

You can thank me later.

And if you just look at `/etc/systemd/system/sshd.service` you'll feel kind of magic as well as there's nothing there related to EIC. That is just a symlink to `/lib/systemd/system/ssh`, which has all the meat there.

Other than `sshd.service`, `syslog.service` also symlink to `/lib/systemd/system/rsyslog.service`.

Ok, so `/etc/systemd/system` is supposed to be the place where you want to override the system's service unit definition, which is usually in `/lib/systemd/system`. `/etc/systemd/system` will have precedence over `/lib/systemd/system`.

And the correct way to override is not by editing the `.service` file but instead by creating a directory called `servicename.service.d/` directory at the same level and include `*.conf` file in that directory. Within that `.conf` file you can override any individual service section attributes such as `ExecStart=`.

So for example in `/lib/systemd/system/ssh.service.d/ec2-instance-connect.conf`, `ExecStart` is overriden with this command instead:-

```bash
[Service]
ExecStart=
ExecStart=/usr/sbin/sshd -D -o "AuthorizedKeysCommand /usr/share/ec2-instance-connect/eic_run_authorized_keys %%u %%f" -o "AuthorizedKeysCommandUser ec2-instance-connect" $SSHD_OPTS
```

If you just look in `/etc/ssh/sshd_config` or `/etc/systemd/system/sshd.service`, you will feel like a fool because in sshd\_config, `AuthorizedKeysCommand` is commented:-

```bash
# Expect .ssh/authorized_keys2 to be disregarded by default in future.#AuthorizedKeysFile     .ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none#AuthorizedKeysCommandUser nobody
```

and in /etc/systemd/system/sshd.service, ExecStart looks like this:-

```bash
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
```

Nothing in there suggested the use of EC2 Instance Connect. And systemd selling proposition is standardization 😊

Let me check how Amazon Linux does it. I think this is very Ubuntu/Debian specific.

`ssh` provides `Include /etc/ssh/sshd_config.d/*.conf` as override mechanism but I think a common dilemma faced by package maintainers or distro builders is whether to use the program's specific mechanism or use the system's mechanism.

And I think it's clear package maintainers prefer system mechanisms.

Amazon linux simply use `sshd_config` file.

```bash
AuthorizedKeysCommand /opt/aws/bin/eic_run_authorized_keys %u %f AuthorizedKeysCommandUser ec2-instance-connect
```

Much clearer. no hidden magic.