Uploaded Oracle XRDP disaster Report

Oracle_Linux_XRDP_Incident_Report.md

@@ -0,0 +1,641 @@
+# Oracle Linux XRDP Root Login and GNOME Terminal Incident Report
+
+## Document Information
+
+- **System:** `miclo-bangladesh-vm-5`
+- **Operating System:** Oracle Linux Server 8.10
+- **Environment:** Virtual Machine / OpenStack
+- **Affected Service:** XRDP / GNOME graphical remote access
+- **Incident Type:** Remote GUI session failure and GNOME Terminal startup failure
+- **Status:** Resolved
+- **Date:** 2026-08-17
+
+---
+
+## 1. Executive Summary
+
+An issue was identified on an Oracle Linux 8.10 virtual machine where SSH access was working, but graphical access through XRDP was unstable.
+
+The incident had two distinct symptoms:
+
+1. XRDP login using the `root` account authenticated successfully, but the graphical session closed immediately after login.
+2. After the root XRDP desktop was made operational, GNOME Terminal did not open even though other graphical applications and `xterm` worked.
+
+The investigation confirmed that networking, port 3389, XRDP authentication, TLS, Xorg, and the installed XRDP packages were functioning correctly.
+
+The first failure was caused by the XRDP window-manager startup path for the root account. XRDP was configured to use a per-user startup script named `startwm.sh`, but `/root/startwm.sh` did not exist. XRDP therefore used its default window-manager startup script, which did not correctly initialize the root GNOME session.
+
+The second failure was caused by an inconsistent D-Bus/systemd user-session environment. The GNOME desktop was running on the XRDP X11 display, but `gnome-terminal-server`, which is activated through the root user's D-Bus/systemd user environment, did not initially receive the correct `DISPLAY`, `XAUTHORITY`, and related session variables. As a result, GNOME Terminal failed with `status=10` and `Cannot open display`.
+
+The issue was resolved by creating a root-specific XRDP startup script, using the standard root user D-Bus at `/run/user/0/bus`, and propagating the XRDP X11 environment into the D-Bus/systemd user activation environment.
+
+---
+
+## 2. Initial Symptoms
+
+The server was reachable through SSH, but direct graphical access and XRDP access were problematic.
+
+System verification showed:
+
+```bash
+cat /etc/os-release
+```
+
+Result:
+
+```text
+NAME="Oracle Linux Server"
+VERSION="8.10"
+VERSION_ID="8.10"
+```
+
+The system was correctly configured to boot into graphical mode:
+
+```bash
+systemctl get-default
+```
+
+Result:
+
+```text
+graphical.target
+```
+
+GNOME Display Manager was active:
+
+```bash
+systemctl status gdm
+```
+
+Result:
+
+```text
+Active: active (running)
+```
+
+XRDP packages were installed:
+
+```bash
+rpm -qa | grep -Ei 'xrdp|xorgxrdp'
+```
+
+Result:
+
+```text
+xrdp-0.10.6.1-3.el8.x86_64
+xrdp-selinux-0.10.6.1-3.el8.x86_64
+xorgxrdp-0.10.5-1.el8.x86_64
+```
+
+---
+
+## 3. XRDP Root Login Failure
+
+### 3.1 Authentication Worked
+
+The XRDP session manager logs confirmed that root credentials were accepted:
+
+```text
+Access permitted for user: root
+Received sys login status for root : logged in
+login was successful - creating session
+```
+
+This ruled out:
+
+- Incorrect root password
+- PAM authentication failure
+- XRDP access control failure
+
+### 3.2 Xorg Started Successfully
+
+XRDP also successfully created an Xorg session:
+
+```text
+Starting X server on display 10
+X server :10 is working
+session is available on display 10 for user root
+```
+
+The RDP client connected to the Xorg session:
+
+```text
+lib_mod_log_peer: xrdp_pid=... connected to Xorg_pid=...
+```
+
+This ruled out:
+
+- Missing `xorgxrdp`
+- Xorg startup failure
+- XRDP-to-Xorg connection failure
+
+### 3.3 Window Manager Exited Immediately
+
+The critical log entries were:
+
+```text
+Window manager finished normally in 1 secs
+Window manager exited quickly
+This could indicate a window manager config problem
+Session on display has finished
+```
+
+This showed that the RDP connection itself was functional, but the GNOME/window-manager process terminated immediately.
+
+---
+
+## 4. Root Cause of the Root XRDP Session Failure
+
+XRDP debug logging showed:
+
+```text
+The user home directory window manager configuration is enabled but window manager program does not exist: /root/startwm.sh
+Using the default window manager on display 10: /usr/libexec/xrdp/startwm-bash.sh
+```
+
+The root cause was therefore the missing root-specific XRDP startup script:
+
+```text
+/root/startwm.sh
+```
+
+XRDP fell back to:
+
+```text
+/usr/libexec/xrdp/startwm-bash.sh
+```
+
+The fallback script did not correctly initialize a persistent GNOME desktop for the root XRDP session.
+
+---
+
+## 5. Initial Remediation for Root XRDP
+
+A root-specific XRDP startup script was created to explicitly start the GNOME session.
+
+Example initial configuration:
+
+```bash
+#!/bin/bash
+
+export XDG_SESSION_TYPE=x11
+export GDK_BACKEND=x11
+
+exec /usr/bin/gnome-session
+```
+
+Permissions and SELinux context were applied:
+
+```bash
+chmod 700 /root/startwm.sh
+restorecon -v /root/startwm.sh
+```
+
+XRDP services were restarted:
+
+```bash
+systemctl restart xrdp-sesman xrdp
+```
+
+After this change, the root user was able to log in to the graphical XRDP desktop.
+
+---
+
+## 6. GNOME Terminal Failure After Root Desktop Login
+
+After the desktop became functional, GNOME Terminal still did not open.
+
+A working `xterm` session was used for troubleshooting.
+
+The XRDP graphical environment contained a valid X11 display similar to:
+
+```text
+DISPLAY=:10.0
+```
+
+The system locale was also verified as UTF-8:
+
+```text
+LANG=en_US.UTF-8
+LC_CTYPE=en_US.UTF-8
+```
+
+GNOME Terminal was then tested through its user service.
+
+### 6.1 GNOME Terminal Service Failure
+
+The service status showed:
+
+```text
+gnome-terminal-server.service - GNOME Terminal Server
+Active: failed
+```
+
+The service exited with:
+
+```text
+status=10
+```
+
+The system journal provided the key failure:
+
+```text
+Unable to init server: Could not connect: Connection refused
+Failed to parse arguments: Cannot open display:
+```
+
+This confirmed that `gnome-terminal-server` could not connect to the XRDP X display.
+
+---
+
+## 7. D-Bus and systemd User Environment Investigation
+
+The root user runtime directory was present:
+
+```bash
+ls -l /run/user/0/bus
+```
+
+Result:
+
+```text
+srw-rw-rw- 1 root root ... /run/user/0/bus
+```
+
+The root systemd user manager was also running:
+
+```bash
+systemctl --user status
+```
+
+Result:
+
+```text
+State: running
+```
+
+However, the systemd user environment initially contained:
+
+```text
+LANG=en_US.UTF-8
+XDG_RUNTIME_DIR=/run/user/0
+```
+
+but did not contain:
+
+```text
+DISPLAY=:10.0
+XAUTHORITY=/root/.Xauthority
+```
+
+This was significant because `gnome-terminal-server` is activated through the root user's D-Bus/systemd user environment.
+
+The GNOME session and systemd user services were therefore operating with inconsistent session information.
+
+---
+
+## 8. D-Bus Session Conflict
+
+An earlier XRDP startup configuration used:
+
+```bash
+dbus-run-session /usr/bin/gnome-session
+```
+
+This created a separate temporary D-Bus session with an address similar to:
+
+```text
+unix:abstract=/tmp/dbus-...
+```
+
+At the same time, the root systemd user manager used the standard user bus:
+
+```text
+unix:path=/run/user/0/bus
+```
+
+The resulting architecture was effectively:
+
+```text
+XRDP / GNOME session
+        |
+        +-- temporary dbus-run-session bus
+
+systemd --user
+        |
+        +-- /run/user/0/bus
+```
+
+GNOME Terminal is implemented through `gnome-terminal-server`, which relies on the user's D-Bus/systemd activation environment.
+
+Because the two environments were not synchronized, the terminal server did not know which X11 display to use.
+
+---
+
+## 9. Final Root Cause
+
+The final root cause was a combination of:
+
+1. Missing `/root/startwm.sh`, causing XRDP to use an unsuitable default window-manager startup path.
+2. A D-Bus/systemd user-session environment that did not contain the XRDP X11 session variables.
+3. Use of `dbus-run-session`, which created a second temporary D-Bus session instead of using the standard root user bus.
+4. `gnome-terminal-server` being started without a valid XRDP `DISPLAY` and `XAUTHORITY`.
+
+The failure path was:
+
+```text
+XRDP authentication succeeds
+        |
+        v
+Xorg starts successfully
+        |
+        v
+GNOME desktop starts
+        |
+        v
+systemd --user / D-Bus does not know XRDP DISPLAY
+        |
+        v
+gnome-terminal-server cannot connect to Xorg
+        |
+        v
+GNOME Terminal fails to open
+```
+
+---
+
+## 10. Final Working Configuration
+
+The final solution was to use the standard root user D-Bus and propagate the XRDP X11 session variables into the D-Bus/systemd activation environment.
+
+Final `/root/startwm.sh`:
+
+```bash
+#!/bin/bash
+
+export XDG_SESSION_TYPE=x11
+export GDK_BACKEND=x11
+
+export XDG_RUNTIME_DIR=/run/user/0
+export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus
+
+export LANG=en_US.UTF-8
+export LC_CTYPE=en_US.UTF-8
+unset LC_ALL
+
+export XAUTHORITY=/root/.Xauthority
+
+dbus-update-activation-environment --systemd     DISPLAY     XAUTHORITY     XDG_RUNTIME_DIR     LANG     LC_CTYPE
+
+exec /usr/bin/gnome-session
+```
+
+Permissions:
+
+```bash
+chmod 700 /root/startwm.sh
+restorecon -v /root/startwm.sh
+```
+
+XRDP services were restarted:
+
+```bash
+systemctl restart xrdp-sesman xrdp
+```
+
+---
+
+## 11. Validation
+
+After applying the final configuration:
+
+- Root authentication through XRDP succeeded.
+- The XRDP Xorg display started successfully.
+- GNOME desktop loaded successfully.
+- The root systemd user environment received the XRDP display information.
+- GNOME Terminal opened successfully.
+- `gnome-terminal-server` was able to connect to the active XRDP X11 display.
+- Normal XRDP desktop functionality was restored.
+
+Useful validation commands:
+
+```bash
+echo "$DISPLAY"
+echo "$DBUS_SESSION_BUS_ADDRESS"
+echo "$XAUTHORITY"
+echo "$XDG_RUNTIME_DIR"
+```
+
+Expected values are similar to:
+
+```text
+DISPLAY=:10.0
+DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus
+XAUTHORITY=/root/.Xauthority
+XDG_RUNTIME_DIR=/run/user/0
+```
+
+Check the user activation environment:
+
+```bash
+systemctl --user show-environment | grep -E '^(DISPLAY|XAUTHORITY|XDG_RUNTIME_DIR|LANG|LC_CTYPE)='
+```
+
+---
+
+## 12. Services and Components Involved
+
+| Component | Purpose | Status During Incident |
+|---|---|---|
+| SSH | Remote command-line administration | Working |
+| GDM | GNOME Display Manager | Working |
+| XRDP | RDP protocol server | Working |
+| xrdp-sesman | XRDP session manager | Working |
+| xorgxrdp | Xorg backend for XRDP | Working |
+| Xorg | Graphical display server | Working |
+| GNOME Session | Graphical desktop environment | Initially failed for root |
+| D-Bus | IPC/session activation | Misaligned session environment |
+| systemd --user | Per-user service manager | Running but missing XRDP display variables |
+| gnome-terminal-server | GNOME Terminal backend | Failed to connect to X display |
+| xterm | Basic X11 terminal | Working |
+
+---
+
+## 13. Important Log Indicators
+
+### Successful authentication
+
+```text
+Access permitted for user: root
+Received sys login status for root : logged in
+```
+
+### Successful Xorg startup
+
+```text
+Starting X server on display 10
+X server :10 is working
+```
+
+### Initial desktop failure
+
+```text
+Window manager exited quickly
+This could indicate a window manager config problem
+```
+
+### Missing user startup script
+
+```text
+The user home directory window manager configuration is enabled
+but window manager program does not exist: /root/startwm.sh
+```
+
+### GNOME Terminal failure
+
+```text
+gnome-terminal-server.service: Main process exited
+status=10
+```
+
+### Actual terminal-server error
+
+```text
+Unable to init server: Could not connect: Connection refused
+Failed to parse arguments: Cannot open display:
+```
+
+---
+
+## 14. Troubleshooting Commands Used
+
+### XRDP status
+
+```bash
+systemctl status xrdp
+systemctl status xrdp-sesman
+```
+
+### XRDP packages
+
+```bash
+rpm -qa | grep -Ei 'xrdp|xorgxrdp'
+```
+
+### Graphical target
+
+```bash
+systemctl get-default
+```
+
+### GNOME Display Manager
+
+```bash
+systemctl status gdm
+```
+
+### XRDP logs
+
+```bash
+tail -300 /var/log/xrdp.log
+tail -300 /var/log/xrdp-sesman.log
+```
+
+### Xorg XRDP logs
+
+```bash
+ls -ltr /root/.xorgxrdp*
+```
+
+### Channel logs
+
+```bash
+find /root /var/log -type f -name 'xrdp-chansrv*.log'
+```
+
+### GNOME Terminal status
+
+```bash
+systemctl --user status gnome-terminal-server.service
+```
+
+### GNOME Terminal service exit information
+
+```bash
+systemctl --user show gnome-terminal-server.service     -p Result     -p ExecMainCode     -p ExecMainStatus     -p ExecStart
+```
+
+### System journal
+
+```bash
+journalctl -b --no-pager | grep -Ei 'gnome-terminal|terminal-server'
+```
+
+### Root user D-Bus
+
+```bash
+ls -l /run/user/0/bus
+```
+
+### Root systemd user environment
+
+```bash
+systemctl --user show-environment
+```
+
+---
+
+## 15. Preventive Actions
+
+1. Keep a documented root-specific XRDP startup configuration if root graphical access is operationally required.
+2. Avoid creating an unnecessary second D-Bus session with `dbus-run-session` when a systemd user bus is already available.
+3. Ensure XRDP session variables are propagated to the systemd/D-Bus activation environment:
+   - `DISPLAY`
+   - `XAUTHORITY`
+   - `XDG_RUNTIME_DIR`
+   - `LANG`
+   - `LC_CTYPE`
+4. Keep XRDP debug logging available for troubleshooting, but reduce verbosity after resolution.
+5. Monitor `/var/log/xrdp.log` and `/var/log/xrdp-sesman.log` for session startup failures.
+6. Back up `/etc/xrdp/` and `/root/startwm.sh` after confirming the working configuration.
+7. Prefer standard administrative users with privilege escalation where possible, because graphical root sessions are atypical and can create application compatibility issues.
+
+---
+
+## 16. Backup Recommendation
+
+Back up the working XRDP configuration:
+
+```bash
+mkdir -p /root/xrdp-config-backup
+
+cp -a /etc/xrdp /root/xrdp-config-backup/
+cp -a /root/startwm.sh /root/xrdp-config-backup/
+```
+
+Optional archive:
+
+```bash
+tar -czf /root/xrdp-working-config-$(date +%F).tar.gz     /etc/xrdp     /root/startwm.sh
+```
+
+---
+
+## 17. Incident Resolution Status
+
+**Resolved**
+
+Final service state:
+
+```text
+SSH                      Working
+XRDP                     Working
+Root XRDP authentication Working
+Xorg XRDP session        Working
+GNOME desktop            Working
+D-Bus user session       Working
+systemd --user           Working
+GNOME Terminal           Working
+```
+
+The final resolution was achieved by aligning the XRDP X11 graphical session with the root user's standard D-Bus and systemd user-service environment.