By Raushan Raj, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.
How can we build an endpoint security solution for our machines deployed on Alibaba Cloud? Do you want to setup HIDS (Host Intrusion Detection System) for your organization? If you are looking for answers, then this article is for you.
While walking through this article you will first learn about basics of Osquery and using it on Alibaba Cloud Elastic Compute Service (ECS). At the end of the article, you will come to know how easy it is to manage it-compliance, vulnerability management & incident response and to detect intrusions on machines using an open source solution without investing a penny on commercial solutions.
Osquery is an operating system instrumentation framework that exposes an operating system as a high-performance relational database. This allows you to write SQL queries to explore operating system data.
Osquery is a perfect tool for HIDS once it is configured properly as it has the power to monitor thousands of machines simultaneously. Adding analytics and alerts on top of osquery logs will help in the easy setup of in-house EDR Solution.
Alternatively, osquery is an agent that will sit on your machines (Linux, Windows, Mac) and will transfer logs to your central server for security analytics and monitoring. Osquery treats your machines as a SQL database and subsequently provides SQL based query syntax to easily gather information out of it. You may use osquery as
1. osqueryd: Daemon, It will execute your queries in the background and save results to log source.
2. osqueryi : Interactive shell, You can execute queries on the shell and get results there only.
Don't worry we are going to discuss what queries are all about in further sections.
Run the following command to install and run osquery with default configurations.
$ export OSQUERY_KEY=1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys $OSQUERY_KEY
$ sudo add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb deb main'
$ sudo apt-get update
$ sudo apt-get install osquery
$ sudo service osqueryd restartOsquery has built in tables like logged_in_users, cpu_time, process_events, file_events, etc.
It understands SQL query syntax. Below is the query to get 5 processes running in a system.
root@home:~$ osqueryi
Using a virtual database. Need help, type '.help'
osquery> select pid, name, path, cmdline, system_time from processes limit 5;
+------+-----------------+------+----------------------------------------+-------------+
| pid | name | path | cmdline | system_time |
+------+-----------------+------+----------------------------------------+-------------+
| 1 | systemd | | /sbin/init splash | 3810 |
| 10 | rcu_sched | | | 8470 |
| 1025 | rabbitmq-server | | /bin/sh /usr/lib/rabbitmq/bin/rabbitmq | 0 |
| 1051 | redis-server | | /usr/bin/redis-server 127.0.0.1:6379 | 9020 |
| 106 | kthrotld | | | 0 |
+------+-----------------+------+----------------------------------------+-------------+1. /etc/osquery/osquery.conf: Configuration file follows the below syntax.
{
"options" {...},
"schedule"{
"deb_packages":{
"query" : "select * from deb_packages;"
"interval" : "86400",
"snapshot" : true,
"platform": "linux",
"description" : "Fetches all deb packages installed in system"
},
"query_name_2":{
...
}
},
"packs":{
"pack1" : "/path/to/pack1.conf",
"pack2" : "/path/to/pack2/conf"
...
},
"file_paths":{
"configurations":[
"/etc/%%",
"/bin/%%",
]
}
}options: osqueryd --help will give you all possible options key: value with usage. You may leave it empty. "options": {
"events_max": 100000,
"enable_monitor": true,
"host_identifier": "uuid"
}* `schedule`: schedule block contains a list of all the queries you want to execute. Query block has the following keys:
1. _query_: SQL query you need to execute on a system level. In the first query, we are fetching deb_packages installed on the system.
2. _interval_: Time in seconds to execute the query next time.
3. _snapshot_: If the snapshot is False then only added or removed deb packages will be returned over time, else whole deb packages will be returned over an interval.
4. _platform_: Queries needs to be defined platform specific i.e; Linux, POSIX, Darwin, Windows, FreeBSD
* `packs`: Packs are a collection of queries intended for the specific purpose. Example: A "vulnerability management" pack may perform general asset management queries that build event logs around package and software install changes. _pack.conf_ looks like below{
"queries":{
"query_name_1":{
...
},
...
}
}* `file_paths`: This block contains path of files/directory which you want to monitor for addtion, removal, access changes, etc. The changes are received in `file_events` table, hence _select * from file_events;_ query is required in schedule block. Generally, any table with `_events` suffix like _user_events_,_process_events_,_socket_events_ are populated using pubsub framework only when events got triggered.
2. /etc/osquery/osquery.flag: It is the default path of --flagfile args while running osqueryd. It provides settings required to initiate osqueryd.
--config_plugin = filesystem
--logger_plugin = filesystem
--logger_path = /var/log/osquery
--host_identifier = uuid
--utc=true* `config_plugin <tls|filesystem>`: Configuration can be fetched via tls also.
* `logger_plugin <tls|filesystem>` : Output of the queries can be written to tls also.
* `logger_path`: In case of filesystem, path of directory to write logs.
1. Osquery GitHub repo contains predefined packs. Copy the conf file and uncomment the packs required in the file and restart the osqueryd.
$ cp /usr/share/osquery/osquery.example.conf /etc/osquery/osquery.conf "packs": {
"incident-response": "/usr/share/osquery/packs/incident-response.conf",
"it-compliance": "/usr/share/osquery/packs/it-compliance.conf",
"osx-attacks": "/usr/share/osquery/packs/osx-attacks.conf",
"vuln-management": "/usr/share/osquery/packs/vuln-management.conf",
"hardware-monitoring": "/usr/share/osquery/packs/hardware-monitoring.conf",
"ossec-rootkit": "/usr/share/osquery/packs/ossec-rootkit.conf",
},2. Alternativley, download the recommended config and flag files from here. in /etc/osquery/ directory and restart the osqueryd.
In the context of the recommended configuration above, Below is the explanation of security packs with some examples.
Every organization needs to meet certain security standards. This pack will include queries that check for changes in locked down operating system features and user settings. It will also help the organization with some of the GDPR, PCI DSS requirements.
"disk_encryption": {
"query" : "select * from disk_encryption;",
"interval" : "86400",
"version" : "1.4.5",
"platform" : "posix",
"description" : "Retrieves the current disk encryption status for the target system.",
"value" : "Identifies a system potentially vulnerable to disk cloning."
},
"chrome_extensions": {
"query" : "select * from users join chrome_extensions using (uid);",
"interval" : "86400",
"version" : "1.4.5",
"description" : "Retrieves the list of extensions for Chrome in the target system.",
"value" : "General security posture."
},Vulnerability management is the process of identifying security holes in software and then patching them to prevent unwanted access to sensitive systems and data. A vulnerability management pack may perform general asset management queries that build event logs around package and software install changes.
"backdoored_python_packages": {
"query" : "select name as package_name, version as package_version, path as package_path from python_packages where package_name = 'acqusition' or package_name = 'apidev-coop' or package_name = 'bzip' or package_name = 'crypt' or package_name = 'django-server' or package_name = 'pwd' or package_name = 'setup-tools' or package_name = 'telnet' or package_name = 'urlib3' or package_name = 'urllib';",
"interval" : "86400",
"platform" : "posix",
"version" : "1.4.5",
"description" : "Watches for the backdoored Python packages installed on system. See (http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/index.html)",
"value" : "Gives some assurances that no bad Python packages are installed on the system."
}Your system might got infected with malware or attackers may have backdoor established. Main goal of incident response is to identify intrusions and take action on time to reduce the risk of future incidents.This pack will help you with detecting such anomalies.
"logged_in_users": {
"query" : "select liu.*, p.name, p.cmdline, p.cwd, p.root from logged_in_users liu, processes p where liu.pid = p.pid;",
"interval" : "3600",
"platform": "posix",
"version" : "1.4.5",
"description" : "Retrieves the list of all the currently logged in users in the target system.",
"value" : "Useful for intrusion detection and incident response. Verify assumptions of what accounts should be accessing what systems and identify machines accessed during a compromise."
},
"open_sockets": {
"query" : "select distinct pid, family, protocol, local_address, local_port, remote_address, remote_port, path from process_open_sockets where path <> '' or remote_address <> '';",
"interval" : "86400",
"platform": "posix",
"version" : "1.4.5",
"description" : "Retrieves all the open sockets per process in the target system.",
"value" : "Identify malware via connections to known bad IP addresses as well as odd local or remote port bindings"
},All the logs generated are in JSON format
These are differential changes between the last (most recent) query execution and the current execution. Each log line is a JSON string that indicates what data has been added/removed by which query.
{
"action": "added",
"columns": {
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97830"
},
"name": "processes",
"hostname": "hostname.local",
"calendarTime": "Tue Sep 30 17:37:30 2014",
"unixTime": "1412123850",
"epoch": "314159265",
"counter": "1"
}
{
"action": "removed",
"columns": {
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97650"
},
"name": "processes",
"hostname": "hostname.local",
"calendarTime": "Tue Sep 30 17:37:30 2014",
"unixTime": "1412123850",
"epoch": "314159265",
"counter": "1"
}A snapshot is an 'exact point in time' set of results, no differentials. If you always want a list of mounts, not the added and removed mounts, use a snapshot.
{
"action": "snapshot",
"snapshot": [
{
"parent": "0",
"path": "/sbin/launchd",
"pid": "1"
},
{
"parent": "1",
"path": "/usr/sbin/syslogd",
"pid": "51"
},
{
"parent": "1",
"path": "/usr/libexec/UserEventAgent",
"pid": "52"
},
{
"parent": "1",
"path": "/usr/libexec/kextd",
"pid": "54"
}
],
"name": "process_snapshot",
"hostIdentifier": "hostname.local",
"calendarTime": "Mon May 2 22:27:32 2016 UTC",
"unixTime": "1462228052",
"epoch": "314159265",
"counter": "1"
}
Now that we know how to configure osquery and get the output of queries in the log file. In the coming articles we are going to learn managing osquery on Alibaba's Machine fleets, learn how to use tls plugins, parsing logs with logstash and ingesting it into ElasticSearch, using ElastAlert for the alert on security incidents, enabling threat hunting, etc.
How to Deploy a SQL Server Container in Alibaba Container Service Kubernetes
2,599 posts | 764 followers
FollowAlibaba Clouder - February 20, 2021
Cheng - February 7, 2022
Alibaba Clouder - April 7, 2021
JJ Lim - December 13, 2021
Alibaba Clouder - December 29, 2020
Alibaba Clouder - February 24, 2020
2,599 posts | 764 followers
Follow
ECS(Elastic Compute Service)
Elastic and secure virtual cloud servers to cater all your cloud hosting needs.
Learn More
WAF(Web Application Firewall)
A cloud firewall service utilizing big data capabilities to protect against web-based attacks
Learn More
OSS(Object Storage Service)
An encrypted and secure cloud storage service which stores, processes and accesses massive amounts of data from anywhere in the world
Learn MoreMore Posts by Alibaba Clouder