You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 01:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Fri Jan 02, 2026 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2583045997422,12cyclictest309-21in:imuxsock12:39:593
2583045997226,10cyclictest2597976-21systemctl12:00:003
2583036997229,39cyclictest1-21systemd07:34:591
258303699701,14cyclictest2595871-21cron11:17:001
2583045996926,12cyclictest275-21dbus-daemon11:19:593
258303699699,9cyclictest2597190-21sshd11:43:391
2583045996824,5cyclictest2583776-21latency_hist07:25:013
2583045996622,10cyclictest0-21swapper/309:15:013
258303699661,12cyclictest2588366-21sshd08:50:411
258303699652,15cyclictest2595396-21munin-run11:10:011
258303699652,12cyclictest2594596-21sshd10:54:501
258303699651,13cyclictest2599501-21sshd12:25:391
258303699641,12cyclictest2591802-21munin-run10:00:011
258303399641,3cyclictest131rcu_preempt08:25:020
258303699632,16cyclictest2599796-21munin-run12:35:011
258303699632,16cyclictest2589453-21munin-run09:15:001
258303699632,14cyclictest2583769-21munin-run07:25:021
258303699631,16cyclictest2584548-21munin-run07:40:011
258303699631,15cyclictest2598461-21sshd12:05:401
258303699631,14cyclictest2592053-21munin-run10:05:011
258303699631,14cyclictest2592053-21munin-run10:05:001
2583045996229,12cyclictest37-21ksoftirqd/309:25:003
2583045996220,12cyclictest2593585-21systemd-run10:35:013
258303699622,13cyclictest2593343-21munin-run10:30:011
258303699622,13cyclictest2584800-21munin-run07:45:021
258303699621,9cyclictest2583458-21sshd07:19:041
258303699621,15cyclictest2589993-21munin-run09:25:011
258303699621,13cyclictest2590993-21sshd09:44:411
2583045996121,8cyclictest2592468-21cat10:10:023
2583045996120,13cyclictest275-21dbus-daemon07:55:013
2583045996120,13cyclictest275-21dbus-daemon07:55:013
2583036996139,16cyclictest1-21systemd08:20:011
258303699613,6cyclictest2586071-21munin-run08:10:001
2583036996132,25cyclictest2597125-21latency_hist11:40:021
2583036996129,29cyclictest2597946-21sshd11:59:391
258303699612,16cyclictest2589742-21munin-run09:20:021
2583036996032,24cyclictest238-21systemd-journal11:54:591
258303699602,14cyclictest2588929-21munin-run09:05:001
2583036996020,36cyclictest2585287-21latency_hist07:55:031
2583036996020,36cyclictest2585287-21latency_hist07:55:021
2583033996013,39cyclictest12-21ksoftirqd/008:55:000
2583045995941,13cyclictest238-21systemd-journal08:49:593
258303699597,4cyclictest0-21swapper/109:09:401
2583036995922,32cyclictest2592461-21latency_hist10:10:021
258303699592,15cyclictest2584006-21munin-run07:30:011
258303699592,14cyclictest364-21snmpd08:03:391
2583036995921,34cyclictest2587629-21/usr/sbin/munin08:40:011
258303699592,12cyclictest2593592-21munin-run10:35:011
258303699591,14cyclictest2597432-21sshd11:45:391
258303699591,14cyclictest2597432-21sshd11:45:391
258303399595,17cyclictest238-21systemd-journal08:40:000
258303699589,4cyclictest0-21swapper/108:21:261
2583036995813,41cyclictest2594012-21cut10:40:011
258303699581,11cyclictest2588163-21/usr/sbin/munin08:50:011
258303699572,14cyclictest2598806-21latency_hist12:15:011
258303699572,12cyclictest275-21dbus-daemon10:50:011
258303699572,12cyclictest2585524-21sshd07:59:401
2583036995712,41cyclictest0-21swapper/110:11:481
258303699571,13cyclictest364-21snmpd12:16:381
258303699571,12cyclictest2596660-21sshd11:32:411
258303699567,4cyclictest0-21swapper/111:28:491
258303699563,7cyclictest2586324-21munin-run08:15:011
258303699562,8cyclictest2600020-21cron12:40:001
258303699561,16cyclictest364-21snmpd07:49:171
258303699561,16cyclictest364-21snmpd07:49:161
258303699561,12cyclictest2587088-21modprobe08:27:411
258303699561,12cyclictest2587088-21modprobe08:27:411
258303399561,4cyclictest131rcu_preempt11:50:020
258303399561,4cyclictest131rcu_preempt11:50:010
258303699552,10cyclictest2599259-21modprobe12:22:391
258303699552,10cyclictest2599259-21modprobe12:22:391
2583033995542,9cyclictest2598990-21cron12:20:000
258303699543,7cyclictest364-21snmpd12:03:311
258303699541,10cyclictest2594835-21sshd10:57:401
258303699531,12cyclictest364-21snmpd11:11:301
258303699531,12cyclictest364-21snmpd11:11:301
258303699531,12cyclictest2587878-21sshd08:44:251
2583036995310,39cyclictest0-21swapper/107:12:571
2583036995310,39cyclictest0-21swapper/107:12:561
2583033995319,31cyclictest2584509-21latency_hist07:40:010
2583045995234,14cyclictest275-21dbus-daemon07:59:593
258303699523,6cyclictest2590449-21sshd09:31:401
258303699523,6cyclictest2590449-21sshd09:31:391
258303699521,9cyclictest2590199-21modprobe09:26:401
258303699521,8cyclictest2592774-21sshd10:17:411
2583036995215,34cyclictest275-21dbus-daemon10:45:001
258303699521,13cyclictest364-21snmpd08:34:591
258303399525,3cyclictest131rcu_preempt08:10:020
2583045995120,11cyclictest37-21ksoftirqd/309:10:023
258303699514,6cyclictest2583040-21latency_hist07:10:021
258303699511,9cyclictest414-21sshd09:52:401
2583036995110,12cyclictest2588631-21sshd08:59:261
2583033995147,2cyclictest0-21swapper/011:25:020
2583033995136,11cyclictest238-21systemd-journal10:25:010
258303699501,9cyclictest2590766-21true09:40:001
2583036995011,33cyclictest309-21in:imuxsock10:25:011
2583033995043,4cyclictest16950irq/224-800000009:45:010
2583033995042,5cyclictest16950irq/224-800000008:50:000
258303399501,3cyclictest0-21swapper/012:30:010
2583045994933,10cyclictest275-21dbus-daemon09:30:013
258304199495,3cyclictest131rcu_preempt11:00:002
258304199494,4cyclictest131rcu_preempt12:40:012
258303699499,10cyclictest2591232-21sshd09:47:391
258303399496,8cyclictest12-21ksoftirqd/007:20:000
2583033994917,28cyclictest2595690-21latency_hist11:15:020
2583033994917,28cyclictest2595690-21latency_hist11:15:020
258304199484,5cyclictest131rcu_preempt08:25:012
258304199482,3cyclictest131rcu_preempt08:35:022
258303699482,13cyclictest2595177-21cat11:05:011
258303399485,4cyclictest131rcu_preempt09:00:010
258303399483,7cyclictest2584008-21cut07:30:010
2583045994732,11cyclictest275-21dbus-daemon09:05:003
258304199474,3cyclictest131rcu_preempt07:50:012
258304199474,3cyclictest131rcu_preempt07:50:012
258304199474,3cyclictest131rcu_preempt07:40:002
258304199473,3cyclictest131rcu_preempt10:38:412
258303399474,4cyclictest131rcu_preempt08:05:000
2583045994630,12cyclictest238-21systemd-journal12:15:003
258304199464,3cyclictest131rcu_preempt10:10:012
258304199463,4cyclictest131rcu_preempt08:19:592
258304199463,3cyclictest131rcu_preempt10:49:592
2583041994633,9cyclictest1-21systemd09:55:012
258304199462,5cyclictest131rcu_preempt09:05:012
258304199461,3cyclictest131rcu_preempt10:30:012
258303699462,14cyclictest2596198-21systemd-run11:25:011
258303399465,4cyclictest131rcu_preempt09:20:000
258303399462,4cyclictest131rcu_preempt08:33:170
258303399462,4cyclictest131rcu_preempt07:55:030
258303399462,4cyclictest131rcu_preempt07:55:030
258303399462,40cyclictest0-21swapper/010:35:000
258303399461,3cyclictest131rcu_preempt08:00:010
258304199454,3cyclictest131rcu_preempt12:20:012
258304199453,4cyclictest131rcu_preempt07:15:012
258304199453,4cyclictest131rcu_preempt07:15:002
258304199453,3cyclictest131rcu_preempt09:30:022
2583041994527,14cyclictest238-21systemd-journal11:35:002
258304199452,3cyclictest131rcu_preempt08:10:022
258304199451,3cyclictest131rcu_preempt07:55:032
258304199451,3cyclictest131rcu_preempt07:55:022
258303399453,3cyclictest131rcu_preempt07:15:010
258303399453,3cyclictest131rcu_preempt07:15:000
258303399452,13cyclictest2594609-21latency_hist10:55:010
2583045994427,13cyclictest1-21systemd10:40:013
2583045994426,13cyclictest275-21dbus-daemon11:00:003
258304199445,3cyclictest131rcu_preempt10:30:022
258304199444,3cyclictest131rcu_preempt12:30:002
258304199444,3cyclictest131rcu_preempt11:30:012
258304199444,3cyclictest131rcu_preempt08:40:002
2583041994435,6cyclictest2592830-21sendmail-msp10:20:012
2583041994429,12cyclictest275-21dbus-daemon12:00:012
2583041994426,13cyclictest238-21systemd-journal08:05:002
258304199442,3cyclictest131rcu_preempt09:15:002
258304199441,4cyclictest131rcu_preempt09:15:402
258304199441,3cyclictest131rcu_preempt09:35:032
258304199441,3cyclictest131rcu_preempt09:35:022
2583033994437,4cyclictest16950irq/224-800000009:39:590
2583033994432,7cyclictest2592529-21cron10:15:000
258303399442,3cyclictest131rcu_preempt09:30:400
258303399442,3cyclictest131rcu_preempt09:30:400
258303399442,3cyclictest131rcu_preempt09:05:020
258303399441,3cyclictest131rcu_preempt09:15:010
258304599432,3cyclictest131rcu_preempt10:44:593
258304199439,19cyclictest2595904-21modprobe11:20:002
258304199433,4cyclictest131rcu_preempt07:19:592
2583041994330,8cyclictest238-21systemd-journal08:45:012
258304199432,3cyclictest131rcu_preempt11:20:392
258304199432,3cyclictest131rcu_preempt07:35:012
258304199431,4cyclictest131rcu_preempt10:55:012
258304199431,3cyclictest131rcu_preempt10:45:002
258303399435,4cyclictest131rcu_preempt11:00:020
258303399434,3cyclictest131rcu_preempt11:20:000
258303399434,3cyclictest131rcu_preempt07:50:030
258303399434,3cyclictest131rcu_preempt07:50:020
2583033994335,5cyclictest16850irq/134-800000010:59:590
2583033994333,7cyclictest19350irq/34-mmc111:55:000
2583033994329,11cyclictest2591445-21sed09:50:030
258303399432,3cyclictest131rcu_preempt12:35:000
2583033994319,20cyclictest16850irq/134-800000010:00:400
2583033994319,20cyclictest16850irq/134-800000010:00:390
258303399431,3cyclictest131rcu_preempt11:45:020
258303399431,38cyclictest2597112-21latency_hist11:40:020
2583045994226,11cyclictest275-21dbus-daemon08:10:013
258304199425,8cyclictest30-21ksoftirqd/209:40:002
258304199421,5cyclictest131rcu_preempt09:00:012
258304199421,3cyclictest131rcu_preempt12:35:012
258304199421,3cyclictest131rcu_preempt09:59:392
258303399423,4cyclictest131rcu_preempt11:30:390
258303399423,34cyclictest0-21swapper/012:00:020
2583033994229,10cyclictest12-21ksoftirqd/008:45:020
2583033994228,10cyclictest12-21ksoftirqd/010:00:020
258303399422,5cyclictest131rcu_preempt11:10:010
258303399422,4cyclictest131rcu_preempt07:45:010
258303399422,3cyclictest131rcu_preempt10:50:020
258303399422,3cyclictest131rcu_preempt08:20:010
2583033994215,19cyclictest16850irq/134-800000009:02:400
2583045994126,12cyclictest1-21systemd11:25:013
258304599412,4cyclictest131rcu_preempt09:20:013
2583045994123,13cyclictest275-21dbus-daemon07:35:013
258304199413,4cyclictest131rcu_preempt11:45:022
258304199413,4cyclictest131rcu_preempt08:50:012
258304199413,4cyclictest131rcu_preempt07:25:012
2583041994132,5cyclictest2593067-21cron10:25:012
2583041994127,11cyclictest2585559-21sendmail08:00:002
258304199412,3cyclictest131rcu_preempt11:51:392
258304199412,3cyclictest131rcu_preempt09:44:402
258304199411,3cyclictest131rcu_preempt12:15:002
258304199411,3cyclictest131rcu_preempt10:15:002
258303399415,3cyclictest131rcu_preempt09:27:400
2583033994129,10cyclictest12-21ksoftirqd/009:55:000
258303399411,4cyclictest131rcu_preempt10:09:400
258304599403,5cyclictest131rcu_preempt12:10:023
2583045994026,10cyclictest275-21dbus-daemon08:35:003
2583045994025,10cyclictest238-21systemd-journal11:50:013
2583045994025,10cyclictest238-21systemd-journal11:50:003
2583045994021,14cyclictest37-21ksoftirqd/307:20:023
258304599401,4cyclictest131rcu_preempt11:15:023
258304599401,4cyclictest131rcu_preempt11:15:023
258304199405,9cyclictest30-21ksoftirqd/211:36:352
258304199402,3cyclictest131rcu_preempt11:15:012
258304199402,3cyclictest131rcu_preempt11:15:012
258304199401,3cyclictest131rcu_preempt12:05:402
258304199401,3cyclictest131rcu_preempt11:10:012
2583033994028,9cyclictest12-21ksoftirqd/012:15:010
258303399402,15cyclictest2598497-21modprobe12:10:010
258303399401,3cyclictest131rcu_preempt12:20:050
258303399401,3cyclictest131rcu_preempt12:20:050
2583045993925,10cyclictest275-21dbus-daemon11:45:013
2583045993925,10cyclictest1-21systemd08:55:013
2583045993922,13cyclictest1-21systemd10:05:013
2583045993922,13cyclictest1-21systemd10:05:013
258304599391,3cyclictest131rcu_preempt10:00:023
258304599391,35cyclictest37-21ksoftirqd/308:15:003
2583041993928,8cyclictest1-21systemd11:50:012
2583041993928,8cyclictest1-21systemd11:50:002
258304199392,3cyclictest131rcu_preempt11:05:012
258304199391,3cyclictest131rcu_preempt12:25:022
258304199391,3cyclictest131rcu_preempt12:25:022
258303399393,4cyclictest131rcu_preempt12:39:400
2583033993932,4cyclictest16950irq/224-800000007:35:010
2583033993927,9cyclictest12-21ksoftirqd/009:25:020
2583033993926,9cyclictest238-21systemd-journal12:05:010
258303399391,3cyclictest131rcu_preempt10:30:020
258304599385,4cyclictest131rcu_preempt11:40:003
258304599385,12cyclictest0-21swapper/307:15:003
258304599385,12cyclictest0-21swapper/307:15:003
2583045993829,5cyclictest309-21in:imuxsock09:50:013
258304199383,4cyclictest131rcu_preempt10:05:012
258304199383,4cyclictest131rcu_preempt10:05:012
258304199382,4cyclictest131rcu_preempt07:45:002
2583041993824,10cyclictest238-21systemd-journal07:25:022
258304199381,4cyclictest131rcu_preempt08:54:392
258304199381,3cyclictest131rcu_preempt09:23:412
258304199381,3cyclictest131rcu_preempt08:11:402
258303399387,23cyclictest2593824-21modprobe10:40:000
2583033993815,6cyclictest16850irq/134-800000011:21:410
2583045993726,9cyclictest37-21ksoftirqd/310:55:013
2583045993723,11cyclictest37-21ksoftirqd/309:35:033
2583045993723,10cyclictest37-21ksoftirqd/310:20:003
2583045993721,12cyclictest1-21systemd08:25:013
258304599371,3cyclictest131rcu_preempt12:05:023
258304599371,3cyclictest131rcu_preempt08:20:003
258304199372,21cyclictest2587115-21munin-run08:30:012
258304199372,21cyclictest2587115-21munin-run08:30:002
258304199371,3cyclictest131rcu_preempt09:50:012
258304199371,3cyclictest131rcu_preempt09:09:392
258303399374,3cyclictest131rcu_preempt08:12:390
258303399374,23cyclictest2587116-21grep08:30:010
258303399374,23cyclictest2587116-21grep08:30:000
258303399372,3cyclictest131rcu_preempt10:20:010
258304599362,6cyclictest131rcu_preempt10:25:013
2583045993626,6cyclictest2585820-21munin-plugin-st08:05:003
258304599362,4cyclictest131rcu_preempt12:30:003
2583045993624,9cyclictest37-21ksoftirqd/311:55:003
2583045993624,10cyclictest37-21ksoftirqd/310:50:013
258304599361,3cyclictest131rcu_preempt07:30:023
258304599361,21cyclictest2592542-21munin-run10:15:003
258303399362,3cyclictest131rcu_preempt10:44:590
2583045993527,5cyclictest2595571-21latency_hist11:10:023
2583045993526,5cyclictest2591083-21latency_hist09:45:023
2583045993517,13cyclictest275-21dbus-daemon08:40:013
2583045993516,14cyclictest275-21dbus-daemon07:40:003
258304599351,4cyclictest131rcu_preempt11:00:023
258304599351,4cyclictest131rcu_preempt09:39:593
258304599342,5cyclictest131rcu_preempt12:25:013
258304599342,5cyclictest131rcu_preempt12:25:013
2583045993424,6cyclictest2599013-21munin-plugin-st12:20:003
2583045993421,9cyclictest275-21dbus-daemon11:34:593
258304599341,4cyclictest131rcu_preempt08:30:013
258304599341,4cyclictest131rcu_preempt08:30:003
258304199342,3cyclictest131rcu_preempt12:02:392
2583045993325,5cyclictest2599781-21latency_hist12:35:013
2583045993322,8cyclictest2596643-21sed11:30:033
2583045993318,10cyclictest275-21dbus-daemon09:00:013
258304599331,3cyclictest131rcu_preempt07:10:023
2583041993325,5cyclictest2583072-21latency_hist07:10:022
2583033993328,3cyclictest0-21swapper/007:23:400
258304599322,5cyclictest131rcu_preempt09:54:593
2583045993220,9cyclictest2585078-21latency_hist07:50:023
2583045993220,9cyclictest2585078-21latency_hist07:50:013
258304599313,4cyclictest131rcu_preempt09:35:023
258304599312,11cyclictest1-21systemd08:42:403
258304599311,9cyclictest37-21ksoftirqd/310:30:013
2583045993018,8cyclictest2584867-21cat07:45:023
2583033992923,4cyclictest2582957-21latency_hist07:10:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional