You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-05 - 00:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Sun May 04, 2025 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3911637998177,2cyclictest0-21swapper/012:35:020
3911637998177,2cyclictest0-21swapper/012:35:020
3911637996913,17cyclictest3921340-21munin-run10:20:000
391165199652,16cyclictest3923917-21munin-run11:10:013
391165199652,15cyclictest3919815-21munin-run09:50:023
391165199652,15cyclictest3919815-21munin-run09:50:013
391165199634,13cyclictest3922621-21modprobe10:45:013
391165199634,13cyclictest3922621-21modprobe10:45:013
3911651996316,42cyclictest3913367-21sshd07:41:343
391165199626,11cyclictest272-21dbus-daemon11:00:013
391165199622,7cyclictest3922092-21cron10:35:003
391165199622,15cyclictest3925198-21munin-run11:35:023
391165199622,14cyclictest3927242-21munin-run12:15:013
3911651996214,43cyclictest3927596-21cat12:20:013
391164399628,33cyclictest382-21ntpd10:50:022
3911637996215,13cyclictest3921897-21cat10:30:020
391165199612,54cyclictest3917989-21cron09:14:593
391165199612,14cyclictest3923624-21sshd11:01:343
391165199611,15cyclictest363-21snmpd12:39:473
391165199611,15cyclictest363-21snmpd12:39:473
3911651996111,43cyclictest3911825-21cron07:15:003
391164099616,8cyclictest23-21ksoftirqd/107:10:011
3911651996045,11cyclictest1-21systemd11:50:003
391165199603,7cyclictest3920562-21cron10:05:003
391165199603,7cyclictest3920562-21cron10:05:003
391165199602,11cyclictest3917714-21sshd09:06:353
3911651996015,41cyclictest3925012-21cat11:30:013
3911651996014,39cyclictest1-21systemd12:10:013
3911651996011,44cyclictest0-21swapper/311:52:413
3911651995913,14cyclictest37-21ksoftirqd/309:00:003
391165199591,14cyclictest363-21snmpd10:39:133
391165199591,13cyclictest3915678-21sshd08:29:343
391165199591,13cyclictest363-21snmpd11:35:033
391165199591,11cyclictest3927993-21sshd12:28:343
391164099596,9cyclictest23-21ksoftirqd/110:45:021
391164099596,9cyclictest23-21ksoftirqd/110:45:021
3911640995949,6cyclictest221rcuc/108:40:011
391165199582,9cyclictest3922880-21munin-run10:50:003
391165199582,13cyclictest3924690-21munin-run11:25:013
391165199582,12cyclictest3919519-21sshd09:40:353
3911651995812,41cyclictest3912102-21munin-plugin-st07:20:003
391165199581,12cyclictest363-21snmpd11:44:063
391165199572,15cyclictest3924215-21latency_hist11:15:023
391165199572,10cyclictest363-21snmpd10:21:163
3911651995712,40cyclictest0-21swapper/310:28:523
3911651995712,39cyclictest3919262-21cron09:40:003
391165199571,12cyclictest363-21snmpd08:14:473
3911651995631,21cyclictest272-21dbus-daemon10:09:593
391165199562,11cyclictest3912648-21ls07:30:013
391165199562,10cyclictest3926703-21sshd12:02:343
391165199561,12cyclictest3921288-21modprobe10:15:353
391165199561,12cyclictest3912357-21sshd07:24:343
391165199561,12cyclictest363-21snmpd08:09:063
391165199561,11cyclictest363-21snmpd09:50:093
391165199561,10cyclictest3924368-21sshd11:15:343
3911651995523,27cyclictest3913920-21latency_hist07:55:023
391165199552,15cyclictest3916821-21latency_hist08:50:023
3911651995516,34cyclictest3923217-21cat10:55:023
391165199551,13cyclictest363-21snmpd09:24:143
391164099557,11cyclictest23-21ksoftirqd/108:00:021
3911651995439,10cyclictest1-21systemd12:00:013
391165199542,14cyclictest3916208-21modprobe08:40:013
391165199542,14cyclictest3914431-21unin-run08:05:013
391165199542,11cyclictest363-21snmpd12:24:293
391165199542,11cyclictest363-21snmpd12:24:293
3911643995433,15cyclictest235-21systemd-journal10:20:002
3911637995425,25cyclictest363-21snmpd11:07:230
3911637995420,4cyclictest3927980-21sshd12:25:340
3911637995414,12cyclictest363-21snmpd09:30:020
3911637995413,36cyclictest3924541-21wc11:20:020
391165199534,6cyclictest272-21dbus-daemon10:15:013
391165199534,6cyclictest272-21dbus-daemon10:15:013
391165199532,14cyclictest3928266-21modprobe12:35:003
391165199532,13cyclictest3916970-21munin-run08:55:003
391165199532,13cyclictest3913127-21munin-run07:40:013
391165199532,12cyclictest3915431-21sendmail-mta08:24:243
391165199532,11cyclictest272-21dbus-daemon08:20:013
391165199532,11cyclictest272-21dbus-daemon08:20:013
3911640995331,17cyclictest235-21systemd-journal10:15:001
3911640995331,17cyclictest235-21systemd-journal10:15:001
3911637995333,16cyclictest3922892-21munin-run10:50:010
3911637995332,16cyclictest3915959-21munin-run08:35:000
391165199522,14cyclictest3917461-21sshd09:00:343
3911643995232,15cyclictest235-21systemd-journal11:20:002
391164099523,8cyclictest23-21ksoftirqd/107:30:021
3911637995236,13cyclictest3915417-21sshd08:20:350
3911637995219,30cyclictest3927491-21latency_hist12:20:010
3911637995216,31cyclictest0-21swapper/007:14:340
391165199513,6cyclictest3912872-21grep07:35:003
3911651995127,21cyclictest37-21ksoftirqd/309:35:023
3911651995127,21cyclictest37-21ksoftirqd/309:35:013
391165199512,11cyclictest363-21snmpd09:58:003
391165199511,11cyclictest363-21snmpd07:59:533
3911643995131,15cyclictest235-21systemd-journal11:25:002
3911637995139,8cyclictest3926189-21sshd11:54:330
3911637995136,11cyclictest3916447-21sshd08:42:340
3911637995132,13cyclictest272-21dbus-daemon11:45:010
3911637995131,16cyclictest3927764-21munin-run12:25:010
3911637995131,16cyclictest3927764-21munin-run12:25:010
391164099508,9cyclictest23-21ksoftirqd/112:40:001
391164099508,9cyclictest23-21ksoftirqd/112:40:001
3911640995028,16cyclictest235-21systemd-journal09:50:011
3911640995028,16cyclictest235-21systemd-journal09:50:001
3911640995022,23cyclictest221rcuc/109:45:011
3911637995043,4cyclictest3920823-21munin-run10:09:590
3911637995035,11cyclictest3925185-21munin-run11:35:010
3911637995035,11cyclictest3919769-21sshd09:45:340
3911637995035,11cyclictest3919769-21sshd09:45:330
3911637995034,13cyclictest3922071-21sshd10:30:340
3911637995033,13cyclictest3921056-21sshd10:14:340
3911637995033,13cyclictest3921056-21sshd10:14:340
3911637995033,13cyclictest306-21in:imuxsock11:20:020
3911637995031,14cyclictest3917227-21systemctl09:00:000
3911637995014,33cyclictest0-21swapper/010:45:010
3911637995014,33cyclictest0-21swapper/010:45:000
391165199492,7cyclictest272-21dbus-daemon09:20:013
391165199492,7cyclictest272-21dbus-daemon09:20:013
3911651994924,22cyclictest37-21ksoftirqd/309:30:023
391165199492,14cyclictest3913767-21cut07:50:013
3911651994914,7cyclictest131rcu_preempt08:34:593
3911643994933,12cyclictest235-21systemd-journal10:25:012
391164099496,8cyclictest23-21ksoftirqd/112:15:011
3911640994910,5cyclictest131rcu_preempt11:20:001
3911637994940,6cyclictest3922394-21munin-run10:40:010
3911637994936,10cyclictest3912071-21sshd07:17:340
3911637994932,13cyclictest3925473-21munin-run11:40:010
3911651994816,28cyclictest272-21dbus-daemon08:45:013
3911643994832,10cyclictest3922359-21run-parts10:40:002
391164099487,9cyclictest23-21ksoftirqd/111:15:011
391164099482,13cyclictest23-21ksoftirqd/112:00:011
391164099481,5cyclictest0-21swapper/110:39:591
3911640994811,5cyclictest131rcu_preempt12:10:011
391163799487,37cyclictest0-21swapper/008:10:030
3911637994835,10cyclictest3919533-21sshd09:43:340
3911637994834,11cyclictest3914893-21sshd08:13:330
3911637994834,10cyclictest3914182-21munin-run08:00:010
3911637994833,11cyclictest3921585-21sshd10:23:340
3911637994832,11cyclictest3923106-21sshd10:53:340
3911637994829,14cyclictest3926732-21munin-run12:05:000
3911637994828,14cyclictest272-21dbus-daemon12:10:010
3911651994720,23cyclictest37-21ksoftirqd/307:10:013
3911643994732,7cyclictest30-21ksoftirqd/209:40:002
3911640994737,7cyclictest23-21ksoftirqd/111:40:001
3911640994732,11cyclictest1-21systemd09:10:001
3911637994740,4cyclictest16950irq/224-800000010:00:000
3911637994734,10cyclictest3918216-21sshd09:15:340
3911637994734,10cyclictest3918216-21sshd09:15:340
3911637994734,10cyclictest3912853-21sshd07:34:350
3911637994733,10cyclictest3917507-21munin-run09:05:010
3911637994732,12cyclictest3919013-21sshd09:34:340
3911637994732,12cyclictest3919013-21sshd09:34:330
3911637994730,13cyclictest3924133-21sshd11:13:350
3911637994711,32cyclictest0-21swapper/012:10:340
3911643994625,16cyclictest235-21systemd-journal11:45:012
3911643994625,14cyclictest272-21dbus-daemon07:15:002
3911640994635,7cyclictest3921358-21sendmail-msp10:20:001
391164099462,8cyclictest23-21ksoftirqd/107:36:341
3911637994633,10cyclictest3928229-21sshd12:30:340
3911637994633,10cyclictest3919256-21sshd09:38:340
3911637994633,10cyclictest3916953-21sshd08:54:350
3911637994632,11cyclictest3913086-21sshd07:35:350
3911637994631,11cyclictest3913893-21munin-run07:55:010
3911637994628,13cyclictest363-21snmpd09:05:110
391164399452,9cyclictest131rcu_preempt08:40:002
3911643994528,12cyclictest30-21ksoftirqd/212:25:022
3911643994528,12cyclictest30-21ksoftirqd/212:25:012
391164099457,9cyclictest23-21ksoftirqd/112:05:001
391164099456,9cyclictest23-21ksoftirqd/111:55:011
3911640994510,5cyclictest131rcu_preempt08:35:021
3911637994533,9cyclictest3916701-21sshd08:48:330
3911637994533,8cyclictest3914437-21latency_hist08:05:020
3911637994528,13cyclictest363-21snmpd07:42:230
3911637994527,14cyclictest363-21snmpd11:00:380
3911637994525,15cyclictest3915184-21munin-run08:20:010
3911637994525,15cyclictest3915184-21munin-run08:20:010
3911643994428,13cyclictest3926769-21sed12:05:012
3911640994432,9cyclictest3912079-21sshd07:19:331
3911640994428,10cyclictest23-21ksoftirqd/107:35:001
3911640994418,17cyclictest221rcuc/108:50:011
391164099441,6cyclictest0-21swapper/108:10:351
3911640994415,10cyclictest23-21ksoftirqd/111:10:011
391164099441,36cyclictest306-21in:imuxsock08:43:341
3911640994410,10cyclictest23-21ksoftirqd/110:30:031
391163799447,33cyclictest3911632-21cyclictest12:00:010
3911637994435,4cyclictest3925969-21/usr/sbin/munin11:50:010
3911637994432,8cyclictest0-21swapper/008:40:010
3911637994430,10cyclictest363-21snmpd10:55:350
391163799441,39cyclictest1-21systemd09:15:000
3911643994332,3cyclictest0-21swapper/209:05:012
3911643994329,9cyclictest1-21systemd11:50:012
3911643994326,12cyclictest3920286-21cron10:00:002
3911643994325,12cyclictest235-21systemd-journal09:00:002
391164399431,23cyclictest291rcuc/212:15:022
391164099437,10cyclictest23-21ksoftirqd/111:35:011
3911640994313,6cyclictest131rcu_preempt11:30:011
3911637994334,5cyclictest3913624-21sshd07:48:330
3911637994329,9cyclictest363-21snmpd11:25:350
3911637994322,17cyclictest3911774-21latency_hist07:10:010
3911643994231,7cyclictest306-21in:imuxsock09:35:012
3911643994231,7cyclictest306-21in:imuxsock09:35:002
3911643994231,7cyclictest30-21ksoftirqd/209:50:032
3911643994231,7cyclictest30-21ksoftirqd/209:50:032
3911643994228,10cyclictest306-21in:imuxsock11:15:012
3911643994227,10cyclictest306-21in:imuxsock08:17:002
3911643994227,10cyclictest306-21in:imuxsock08:16:592
391164099424,10cyclictest23-21ksoftirqd/107:45:011
391164099423,7cyclictest23-21ksoftirqd/109:40:011
3911640994232,5cyclictest306-21in:imuxsock09:00:011
3911640994230,8cyclictest235-21systemd-journal08:24:341
3911640994229,10cyclictest23-21ksoftirqd/110:55:001
3911640994212,6cyclictest131rcu_preempt10:35:021
3911640994211,5cyclictest131rcu_preempt07:25:001
3911640994210,6cyclictest131rcu_preempt07:55:031
3911637994224,14cyclictest363-21snmpd07:22:450
3911637994223,15cyclictest12-21ksoftirqd/009:55:010
391164399418,6cyclictest131rcu_preempt07:20:002
3911643994129,9cyclictest30-21ksoftirqd/207:10:022
3911643994121,15cyclictest272-21dbus-daemon10:45:012
3911643994121,15cyclictest272-21dbus-daemon10:45:012
3911640994131,6cyclictest23-21ksoftirqd/112:20:011
3911640994125,12cyclictest23-21ksoftirqd/108:20:021
3911640994125,12cyclictest23-21ksoftirqd/108:20:011
3911640994111,6cyclictest131rcu_preempt09:35:021
3911640994111,6cyclictest131rcu_preempt09:35:021
3911640994110,5cyclictest131rcu_preempt11:50:011
3911637994128,9cyclictest363-21snmpd10:02:030
3911637994128,9cyclictest363-21snmpd10:02:030
3911637994127,10cyclictest363-21snmpd09:22:410
3911640994028,10cyclictest23-21ksoftirqd/110:00:001
3911637994027,9cyclictest363-21snmpd07:25:410
3911637994018,19cyclictest3915667-21sshd08:25:340
3911643993925,10cyclictest3922091-21cron10:34:592
391164099398,11cyclictest23-21ksoftirqd/110:21:581
391164099397,7cyclictest23-21ksoftirqd/109:20:021
391164099397,7cyclictest23-21ksoftirqd/109:20:021
3911640993929,6cyclictest235-21systemd-journal12:30:341
391164099392,7cyclictest23-21ksoftirqd/112:25:021
391164099392,7cyclictest23-21ksoftirqd/112:25:021
3911640993927,8cyclictest1-21systemd11:05:001
3911640993922,11cyclictest272-21dbus-daemon09:25:001
391164099391,9cyclictest0-21swapper/112:29:431
3911640993910,8cyclictest23-21ksoftirqd/108:05:171
3911643993824,10cyclictest1-21systemd10:15:012
3911643993824,10cyclictest1-21systemd10:15:002
3911643993823,11cyclictest30-21ksoftirqd/212:00:022
391164099389,7cyclictest23-21ksoftirqd/110:01:221
391164099389,7cyclictest23-21ksoftirqd/110:01:221
3911640993823,11cyclictest272-21dbus-daemon08:30:011
391164399378,6cyclictest131rcu_preempt09:15:002
391164399373,3cyclictest131rcu_preempt12:35:012
391164399373,3cyclictest131rcu_preempt09:45:022
3911643993728,6cyclictest3912890-21latency_hist07:35:022
3911643993727,6cyclictest271-21cron09:10:012
3911643993724,10cyclictest30-21ksoftirqd/208:45:002
391164399372,3cyclictest131rcu_preempt10:05:012
391164399372,3cyclictest131rcu_preempt10:05:012
3911643993722,10cyclictest306-21in:imuxsock12:20:002
3911643993721,12cyclictest272-21dbus-daemon10:30:022
3911643993720,11cyclictest272-21dbus-daemon08:30:012
3911643993718,14cyclictest235-21systemd-journal09:55:002
391164099377,9cyclictest23-21ksoftirqd/111:40:091
391164099375,8cyclictest23-21ksoftirqd/110:10:021
3911640993730,5cyclictest221rcuc/107:15:011
3911640993729,5cyclictest3914413-21sh08:05:001
3911640993726,8cyclictest235-21systemd-journal10:57:331
3911640993725,8cyclictest23-21ksoftirqd/109:55:011
3911643993629,4cyclictest3913889-21latency_hist07:55:022
3911643993626,6cyclictest1-21systemd08:25:012
3911643993625,9cyclictest30-21ksoftirqd/209:30:012
3911643993624,9cyclictest3916867-21sed08:50:022
3911643993624,9cyclictest30-21ksoftirqd/211:40:002
3911643993623,10cyclictest30-21ksoftirqd/212:30:002
3911643993622,10cyclictest30-21ksoftirqd/211:05:022
3911643993622,10cyclictest272-21dbus-daemon11:00:012
3911643993621,12cyclictest3915929-21sshd08:34:342
3911643993621,10cyclictest30-21ksoftirqd/211:55:022
391164099368,8cyclictest23-21ksoftirqd/109:04:311
3911640993623,9cyclictest3918956-21wc09:30:021
3911643993525,6cyclictest3923923-21latency_hist11:10:022
3911643993522,10cyclictest30-21ksoftirqd/208:10:022
3911640993524,8cyclictest3924802-21sed11:25:031
3911640993522,10cyclictest1-21systemd08:55:001
3911643993426,4cyclictest3914474-21latency_hist08:05:012
3911643993425,7cyclictest30-21ksoftirqd/209:19:352
3911643993425,7cyclictest30-21ksoftirqd/209:19:342
3911643993424,5cyclictest30-21ksoftirqd/210:10:002
3911643993422,9cyclictest3916938-21sshd08:51:342
3911643993420,11cyclictest30-21ksoftirqd/207:25:002
3911643993420,11cyclictest272-21dbus-daemon12:10:012
3911643993418,12cyclictest272-21dbus-daemon12:40:012
3911643993418,12cyclictest272-21dbus-daemon12:40:012
3911643993323,8cyclictest30-21ksoftirqd/207:45:002
3911643993322,8cyclictest30-21ksoftirqd/207:50:012
3911643993322,8cyclictest30-21ksoftirqd/207:40:002
3911643993315,14cyclictest272-21dbus-daemon11:30:012
391164099334,8cyclictest23-21ksoftirqd/110:45:351
3911640993321,9cyclictest3913624-21sshd07:48:341
3911643993218,8cyclictest3918536-21cut09:25:002
391164399314,5cyclictest131rcu_preempt11:35:012
3911643993120,8cyclictest3914937-21wc08:15:012
391164399311,5cyclictest131rcu_preempt08:00:002
391164099315,6cyclictest3917968-21sshd09:10:341
3911643993022,5cyclictest3923236-21latency_hist10:55:012
3911643993012,15cyclictest131rcu_preempt07:30:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional