Configure Docker Ubuntu

admin



  • # docker run -name my-www -d -p 81:80 ubuntu-apache # docker ps Give Docker Container Name Now you can reference the container for manipulation (start, stop, top, stats, etc) only by using the assigned name.
  • There are two versions of Docker – Docker CE (Community Edition) and Docker EE (Enterprise Edition). If you have a small-scale project, or you’re just learning, you will want to use Docker CE. In this tutorial, learn how to install Docker on Ubuntu 18.04.
  • A Docker Swarm, or Docker cluster, is made up of one or more Dockerized hosts that function as manager nodes, and any number of worker nodes. Setting up such a system requires careful manipulation of the Linux firewall. The network ports required for a Docker Swarm to function correctly are: TCP port 2376 for secure Docker client communication.

Docker uses virtualization technology to provide isolated containers for software and tools. This article will show you the easiest way to install Docker on Ubuntu 20.04 and get it running in less than 5 minutes. Step 1: Update System. Ensure your system is updated. Sudo apt -y update Step 2: Install basic dependencies. There are few dependencies we need to configure Docker repositories and do the actual package installation.

-->

You can configure automatic log upload for continuous reports in Cloud App Security using a Docker on an on-premises Ubuntu, Red Hat Enterprise Linux (RHEL), or CentOS server.

Prerequisites

  • OS:

    • Ubuntu 14.04, 16.04, and 18.04
    • RHEL 7.2 or higher
    • CentOS 7.2 or higher

  • Disk space: 250 GB

  • CPU: 2

  • RAM: 4 GB

  • Set your firewall as described in Network requirements

Note

If you have an existing log collector and want to remove it before deploying it again, or if you simply want to remove it, run the following commands:

Log collector performance

The Log collector can successfully handle log capacity of up to 50 GB per hour. The main bottlenecks in the log collection process are:

  • Network bandwidth - Your network bandwidth determines the log upload speed.

  • I/O performance of the virtual machine - Determines the speed at which logs are written to the log collector's disk. The log collector has a built-in safety mechanism that monitors the rate at which logs arrive and compares it to the upload rate. In cases of congestion, the log collector starts to drop log files. If your setup typically exceeds 50 GB per hour, it's recommended that you split the traffic between multiple log collectors.

Set up and configuration

Step 1 – Web portal configuration: Define data sources and link them to a log collector

  1. Go to the Automatic log upload settings page.

    1. In the Cloud App Security portal, click the settings icon followed by Log collectors.

  2. For each firewall or proxy from which you want to upload logs, create a matching data source.

    1. Click Add data source.
    2. Name your proxy or firewall.
    3. Select the appliance from the Source list. If you select Custom log format to work with a network appliance that isn't listed, see Working with the custom log parser for configuration instructions.
    4. Compare your log with the sample of the expected log format. If your log file format doesn't match this sample, you should add your data source as Other.
    5. Set the Receiver type to either FTP, FTPS, Syslog – UDP, or Syslog – TCP, or Syslog – TLS.

    Note

    Integrating with secure transfer protocols (FTPS and Syslog – TLS) often requires additional settings or your firewall/proxy.

    f. Repeat this process for each firewall and proxy whose logs can be used to detect traffic on your network. It's recommended to set up a dedicated data source per network device to enable you to:

    • Monitor the status of each device separately, for investigation purposes.
    • Explore Shadow IT Discovery per device, if each device is used by a different user segment.

  3. Go to the Log collectors tab at the top.

    1. Click Add log collector.
    2. Give the log collector a name.
    3. Enter the Host IP address of the machine you'll use to deploy the Docker. The host IP address can be replaced with the machine name, if there is a DNS server (or equivalent) that will resolve the host name.
    4. Select all Data sources that you want to connect to the collector, and click Update to save the configuration.

  4. Further deployment information will appear. Copy the run command from the dialog. You can use the copy to clipboard icon.

  5. Export the expected data source configuration. This configuration describes how you should set the log export in your appliances.

    Note

    • A single Log collector can handle multiple data sources.
    • Copy the contents of the screen because you will need the information when you configure the Log Collector to communicate with Cloud App Security. If you selected Syslog, this information will include information about which port the Syslog listener is listening on.
    • For users sending log data via FTP for the first time, we recommend changing the password for the FTP user. For more information, see Changing the FTP password.

Step 2 – On-premises deployment of your machine

The following steps describe the deployment in Ubuntu.

Note

The deployment steps for other supported platforms may be slightly different.

  1. Open a terminal on your Ubuntu machine.

  2. Change to root privileges using the command: sudo -i

  3. To bypass a proxy in your network, run the following two commands:

  4. If you accept the software license terms, uninstall old versions and install Docker CE by running the commands appropriate for your environment:

Set

  1. Remove old versions of Docker: yum erase docker docker-engine docker.io

  2. Install Docker engine prerequisites: yum install -y yum-utils

  3. Add Docker repository:

  4. Install Docker engine: yum -y install docker-ce

  5. Start Docker

  6. Test Docker installation: docker run hello-world

  1. Remove old versions of Docker: yum erase docker docker-engine docker.io

  2. Install Docker engine prerequisites:

  3. Add Docker repository:

  4. Install dependencies:

  5. Install Docker engine: sudo yum install docker-ce

  6. Start Docker

  7. Test Docker installation: docker run hello-world

  1. Remove the container-tools module: yum module remove container-tools

  2. Add the Docker CE repository: yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

  3. Modify the yum repo file to use CentOS 8/RHEL 8 packages: sed -i s/7/8/g /etc/yum.repos.d/docker-ce.repo

  4. Install Docker CE: yum install docker-ce

  5. Start Docker

  6. Test Docker installation: docker run hello-world

  1. Remove old versions of Docker: apt-get remove docker docker-engine docker.io

  2. If you are installing on Ubuntu 14.04, install the linux-image-extra package.

  3. Install Docker engine prerequisites:

  4. Verify that the apt-key fingerprint UID is docker@docker.com: apt-key fingerprint | grep uid

  5. Install Docker engine:

  6. Test Docker installation: docker run hello-world

  1. Deploy the collector image on the hosting machine by importing the collector configuration. Import the configuration by copying the run command generated in the portal. If you need to configure a proxy, add the proxy IP address and port number. For example, if your proxy details are 192.168.10.1:8080, your updated run command is:

  2. Verify that the collector is running properly with the following command: docker logs <collector_name>

You should see the message: Finished successfully!

Step 3 - On-premises configuration of your network appliances

Configure your network firewalls and proxies to periodically export logs to the dedicated Syslog port or the FTP directory according to the directions in the dialog. For example:

Step 4 - Verify the successful deployment in the Cloud App Security portal

Check the collector status in the Log collector table and make sure the status is Connected. If it's Created, it's possible the log collector connection and parsing haven't completed.

Image

You can also go to the Governance log and verify that logs are being periodically uploaded to the portal.

Alternatively, you can check the log collector status from within the docker container using the following commands:

  1. Log in to the container by using this command: docker exec -it <Container Name> bash
  2. Verify the log collector status using this command: collector_status -p

If you have problems during deployment, see Troubleshooting Cloud Discovery.

Optional - Create custom continuous reports

Verify that the logs are being uploaded to Cloud App Security and that reports are generated. After verification, create custom reports. You can create custom discovery reports based on Azure Active Directory user groups. For example, if you want to see the cloud use of your marketing department, import the marketing group using the import user group feature. Then create a custom report for this group. You can also customize a report based on IP address tag or IP address ranges.

  1. In the Cloud App Security portal, under the Settings cog, select Cloud Discovery settings, and then select Continuous reports.
  2. Click the Create report button and fill in the fields.
  3. Under the Filters you can filter the data by data source, by imported user group, or by IP address tags and ranges.

Next steps

If you run into any problems, we're here to help. To get assistance or support for your product issue, please open a support ticket.

Estimated reading time: 8 minutes

Docker includes multiple logging mechanisms to help youget information from running containers and services.These mechanisms are called logging drivers. Each Docker daemon has a defaultlogging driver, which each container uses unless you configure it to use adifferent logging driver, or “log-driver” for short.

As a default, Docker uses the json-file logging driver, whichcaches container logs as JSON internally. In addition to using the logging driversincluded with Docker, you can also implement and use logging driver plugins.

Tip: use the “local” logging driver to prevent disk-exhaustion

Configure Docker On Ubuntu

By default, no log-rotation is performed. As a result, log-files stored by thedefault json-file logging driver logging driver can cause a significant amount of disk space to be used for containers that generate muchoutput, which can lead to disk space exhaustion.

Docker keeps the json-file logging driver (without log-rotation) as a defaultto remain backward compatibility with older versions of Docker, and for situationswhere Docker is used as runtime for Kubernetes.

For other situations, the “local” logging driver is recommended as it performslog-rotation by default, and uses a more efficient file format. Refer to theConfigure the default logging driversection below to learn how to configure the “local” logging driver as a default,and the local file logging driver page for more details about the“local” logging driver.

Configure the default logging driver

To configure the Docker daemon to default to a specific logging driver, set thevalue of log-driver to the name of the logging driver in the daemon.jsonconfiguration file. Refer to the “daemon configuration file” section in thedockerd reference manualfor details.

The default logging driver is json-file. The following example sets the defaultlogging driver to the local log driver:

If the logging driver has configurable options, you can set them in thedaemon.json file as a JSON object with the key log-opts. The followingexample sets two configurable options on the json-file logging driver:

Restart Docker for the changes to take effect for newly created containers.Existing containers do not use the new logging configuration.

Docker

Note

log-opts configuration options in the daemon.json configuration file mustbe provided as strings. Boolean and numeric values (such as the value formax-file in the example above) must therefore be enclosed in quotes (').

If you do not specify a logging driver, the default is json-file.To find the current default logging driver for the Docker daemon, rundocker info and search for Logging Driver. You can use the followingcommand on Linux, macOS, or PowerShell on Windows:

Note

Changing the default logging driver or logging driver options in the daemonconfiguration only affects containers that are created after the configurationis changed. Existing containers retain the logging driver options that wereused when they were created. To update the logging driver for a container, thecontainer has to be re-created with the desired options.Refer to the configure the logging driver for a containersection below to learn how to find the logging-driver configuration of acontainer.

Configure the logging driver for a container

When you start a container, you can configure it to use a different loggingdriver than the Docker daemon’s default, using the --log-driver flag. If thelogging driver has configurable options, you can set them using one or moreinstances of the --log-opt <NAME>=<VALUE> flag. Even if the container uses thedefault logging driver, it can use different configurable options.

The following example starts an Alpine container with the none logging driver.

To find the current logging driver for a running container, if the daemonis using the json-file logging driver, run the following docker inspectcommand, substituting the container name or ID for <CONTAINER>:

Configure the delivery mode of log messages from container to log driver

Docker provides two modes for delivering messages from the container to the logdriver:

  • (default) direct, blocking delivery from container to driver
  • non-blocking delivery that stores log messages in an intermediate per-containerring buffer for consumption by driver

The non-blocking message delivery mode prevents applications from blocking dueto logging back pressure. Applications are likely to fail in unexpected ways whenSTDERR or STDOUT streams block.

Warning

When the buffer is full and a new message is enqueued, the oldest message inmemory is dropped. Dropping messages is often preferred to blocking thelog-writing process of an application.

The mode log option controls whether to use the blocking (default) ornon-blocking message delivery.

The max-buffer-size log option controls the size of the ring buffer used forintermediate message storage when mode is set to non-blocking. max-buffer-sizedefaults to 1 megabyte.

The following example starts an Alpine container with log output in non-blockingmode and a 4 megabyte buffer:

Use environment variables or labels with logging drivers

Some logging drivers add the value of a container’s --env|-e or --labelflags to the container’s logs. This example starts a container using the Dockerdaemon’s default logging driver (let’s assume json-file) but sets theenvironment variable os=ubuntu.

Create Docker Ubuntu Container

If the logging driver supports it, this adds additional fields to the loggingoutput. The following output is generated by the json-file logging driver:

Supported logging drivers

The following logging drivers are supported. See the link to each driver’sdocumentation for its configurable options, if applicable. If you are usinglogging driver plugins, you maysee more options.

DriverDescription
noneNo logs are available for the container and docker logs does not return any output.
localLogs are stored in a custom format designed for minimal overhead.
json-fileThe logs are formatted as JSON. The default logging driver for Docker.
syslogWrites logging messages to the syslog facility. The syslog daemon must be running on the host machine.
journaldWrites log messages to journald. The journald daemon must be running on the host machine.
gelfWrites log messages to a Graylog Extended Log Format (GELF) endpoint such as Graylog or Logstash.
fluentdWrites log messages to fluentd (forward input). The fluentd daemon must be running on the host machine.
awslogsWrites log messages to Amazon CloudWatch Logs.
splunkWrites log messages to splunk using the HTTP Event Collector.
etwlogsWrites log messages as Event Tracing for Windows (ETW) events. Only available on Windows platforms.
gcplogsWrites log messages to Google Cloud Platform (GCP) Logging.
logentriesWrites log messages to Rapid7 Logentries.

Note

When using Docker Engine 19.03 or older, the docker logs commandis only functional for the local, json-file and journald logging drivers.Docker 20.10 and up introduces “dual logging”, which uses a local buffer thatallows you to use the docker logs command for any logging driver. Refer toreading logs when using remote logging drivers for details.

Configure Docker Ubuntu 16.04

Limitations of logging drivers

  • Reading log information requires decompressing rotated log files, which causesa temporary increase in disk usage (until the log entries from the rotatedfiles are read) and an increased CPU usage while decompressing.
  • The capacity of the host storage where the Docker data directory residesdetermines the maximum size of the log file information.
docker, logging, driver