# Logs

## Overview

The logging architecture backing HPE Morpheus Enterprise uses the latest and greatest technologies and standards to be able to service large amounts of log traffic as well as facilitate easy viewing. Utilizing Opensearch behind the scenes and buffered log transmission protocols HPE Morpheus Enterprise provides a highly efficient and highly scalable solution for capturing log data from anything provisioned via the system. By utilizing common formats (syslog) it is also very easy to forward logs to external third party log services.

### Configuration

Logging configuration can be setup in the Administration > Settings > Logs section. There are useful settings here, including customizing the retainment policy (7 days by default). This could be expanded to years for PCI compliance purposes or other requirements an organization might have.

NOTE

When increasing the retainment policy of the logging system, it may be necessary to scale out the Opensearch cluster. Please refer to the relevant information with regards to scaling Opensearch and advanced installation options for externalizing the Opensearch cluster.

The Log administration section also provides options for setting custom syslog forward rules. These rules are applied on each individual host therefore keeping the HPE Morpheus Enterprise appliance itself out of the data plane. For information on different syslog formatting rules please refer to the [rsyslog](http://www.rsyslog.com/sending-messages-to-a-remote-syslog-server/) documentation.

### Usage

HPE Morpheus Enterprise automatically sets up and configures logging for all of the standard catalog items provisioned through morpheus. This includes both Docker containers as well as virtual machines. Simply view instance-specific logs in instance detail via the “Logs” tab.

There are several filtering capabilities built into the logging UI with more being added continually. Easily toggle log level filters from the dropdown or change the date range filter using the handy date filter component. A chart is also displayed above logs representing the log counts by level over the selected time range (default last 24 hours). A handy pattern search is also available with some rather capable features based on Lucene search syntax.

TIP

It may be useful to review the [Lucene search query syntax](https://lucene.apache.org/core/2_9_4/queryparsersyntax.html) for powerful use cases.

There are several other places logs can be viewed. Not only can they be viewed across an application in app detail but also across all instances in the account. The main level `Logs` section provides an ability to query all logs produced by the system. It is also possible to view host-specific logs on a docker host by viewing the host detail page via `Infrastructure`.

NOTE

New features are on the roadmap for the main logs section including saved searches, and handy charting dashboards for garnering insights out of log data.

## Exporting Logs

### Log Settings

There are three main log areas in HPE Morpheus Enterprise

- Agent Logs
- HPE Morpheus Enterprise Server Logs
- Activity / Audit Logs

## Agent Logs

When Instances are deployed through HPE Morpheus Enterprise, the installed Agent captures application logs and sends them back to the HPE Morpheus Enterprise server.

In most cases, the built-in HPE Morpheus Enterprise logging features are sufficient for tracking and reviewing Agent logs. However, if needed, HPE Morpheus Enterprise supports integration with advanced logging systems. See the [log integration section](https://docs.morpheusdata.com/en/latest/logs/logging.html#integrations) above for more information.

## HPE Morpheus Enterprise Server Logs

### About this task

The main HPE Morpheus Enterprise application log is in `/var/log/morpheus/morpheus-ui` and the latest log file is named current. This log is archived every 24hrs. There are a number of other log files for the individual infrastructure components as well.

If you wish to export these to an external syslog platform, do the following:

### Procedure

1. Once you have configured your syslog destination (edit rsyslog.conf), create a morpheus-syslog.conf file in the `/etc/rsyslog.d` directory and add the following entries 

   ```
   ```
   module(load="imfile" PollingInterval="10")  
   input(type="imfile" File="/var/log/morpheus/morpheus-ui/current" Tag="morpheus-ui" ReadMode="2" Severity="info" StateFile="morpheus-ui")  
   input(type="imfile" File="/var/log/morpheus/check-server/current" Tag="check-server" ReadMode="2" Severity="info" StateFile="check-server")  
   input(type="imfile" File="/var/log/morpheus/guacd/current" Tag="guacd" ReadMode="2" Severity="info" StateFile="guacd")  
   input(type="imfile" File="/var/log/morpheus/opensearch/current" Tag="opensearch" ReadMode="2" Severity="info" StateFile="opensearch")  
   input(type="imfile" File="/var/log/morpheus/mysql/current" Tag="mysql" ReadMode="2" Severity="info" StateFile="mysql")  
   input(type="imfile" File="/var/log/morpheus/nginx/current" Tag="nginx" ReadMode="2" Severity="info" StateFile="nginx")  
   input(type="imfile" File="/var/log/morpheus/rabbitmq/current" Tag="rabbitmq" ReadMode="2" Severity="info" StateFile="rabbitmq")
   ```
   ```
2. Restart rsyslog

### Results

The log files will now be forwarded to the destination you have defined.

This configuration is valid for an ‘all-in-one’ HPE Morpheus Enterprise server. If the infrastructure components are running on separate servers /clusters, you will need to create the relevant redirects for the logs on those boxes.

## Activity Log

### About this task

The final log type that may require export is the HPE Morpheus Enterprise Activity log. This tracks system changes made by users, for example create and delete instances etc.

### Procedure

1. To set up CEF/SIEM auditing export, you should edit the following file: `logback.xml` located at `/opt/morpheus/conf/logback.xml`.
2. Add the below appender above or below the other appenders in the logback.xml configuration file: 

   ```
   ```
     <appender name="AUDIT" class="ch.qos.logback.core.rolling.RollingFileAppender">  
         <file>/var/log/morpheus/morpheus-ui/audit.log</file>  
         <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">  
             <fileNamePattern>audit.%d{yyyy-MM-dd}.%i.log</fileNamePattern>  
               <maxFileSize>50MB</maxFileSize>  
               <maxHistory>30</maxHistory>  
         </rollingPolicy>  
         <encoder>  
             <pattern>[%d] [%thread] %-5level %logger{15} - %maskedMsg %n</pattern>  
         </encoder>  
     </appender>  
   .. note:: ``maxFileSize`` and ``maxHistory`` values can be updated as needed.
   ```
   ```
3. Add the below logger above or below the other loggers in the logback.xml configuration file (make sure it is below, not above, the appender from the previous step or an error will occur): 

   ```
   ```
   <logger name="com.morpheus.AuditLogService" level="INFO" additivity="false">  
       <appender-ref ref="AUDIT" />  
   </logger>
   ```
   ```
4. Once you have done this, you need to restart the HPE Morpheus Enterprise Application server: 

   ```
   ```
   morpheus-ctl stop morpheus-ui
   ```
   ```

   NOTE

   Please be aware this will stop the web interface for HPE Morpheus Enterprise.
5. Once the service has stopped enter the following at the shell prompt to restart (if the service does not stop, replace stop with graceful-kill and retry) 

   ```
   ```
   morpheus-ctl start morpheus-ui
   ```
   ```
6. To know when the UI is up and running you can run the following command 

   ```
   ```
   morpheus-ctl tail morpheus-ui
   ```
   ```

   Once you see the ASCI art show up you will be able to log back into the User Interface. A new audit file will have been created called audit.log and will found in the default HPE Morpheus Enterprise log path which is `/var/log/morpheus/morpheus-ui/`

### Results

This is only an example and other configurations are possible, such as creating an appender definition for your SIEM audit database product.

## morpheus-ssl nginx logs

NOTE

Morpheus does not put a logrotate in for Morpheus-ssl access logs

svlogd will only rotate the current file, nginx is setup to write the access logs to separate files and not stdout.

Implementation of a log rotate is left up to up to end users for files outside of the services. This is done in case end users have a log management solution.

Below is what a suggested configuration looks like for the file `/etc/logrotate.d/morpheus-nginx`:

```
```
/var/log/morpheus/nginx/morpheus*access.log {  
        daily  
        rotate 14  
        compress  
        delaycompress  
        missingok  
        notifempty  
        create 644 morpheus-app morpheus-app  
        postrotate  
                [ ! -f /var/run/morpheus/nginx/nginx.pid ] || kill -USR1 `cat /var/run/morpheus/nginx/nginx.pid`  
        endscript  
}
```
```
