You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-22 - 17:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Mon Jun 22, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4453489910432,7cyclictest131rcu_preempt10:05:011
4453469910297,3cyclictest0-21swapper/011:30:020
445346999389,2cyclictest0-21swapper/012:15:010
445352997868,7cyclictest30-21ksoftirqd/207:10:022
445346997054,8cyclictest451510-21diskmemload12:30:020
445346996724,35cyclictest12-21ksoftirqd/009:23:110
445354996417,9cyclictest37-21ksoftirqd/312:40:023
445354996415,11cyclictest37-21ksoftirqd/310:11:483
44535499635,21cyclictest493750-21ssh10:43:443
445352996328,7cyclictest30-21ksoftirqd/210:25:012
445354996214,11cyclictest37-21ksoftirqd/308:00:023
44534699628,5cyclictest131rcu_preempt12:06:480
445354996112,11cyclictest37-21ksoftirqd/309:40:393
445346996120,7cyclictest12-21ksoftirqd/009:50:020
445354996023,9cyclictest131rcu_preempt10:40:013
445354995916,8cyclictest37-21ksoftirqd/312:07:003
445354995914,9cyclictest37-21ksoftirqd/312:14:053
445352995915,11cyclictest0-21swapper/212:10:032
44534699599,32cyclictest131rcu_preempt10:37:510
44535499588,10cyclictest487247-21ssh10:29:143
445354995817,7cyclictest37-21ksoftirqd/312:26:263
445354995814,8cyclictest37-21ksoftirqd/307:30:003
445354995812,11cyclictest37-21ksoftirqd/310:05:013
445354995811,9cyclictest37-21ksoftirqd/311:54:133
445354995810,11cyclictest37-21ksoftirqd/312:31:593
445354995810,11cyclictest37-21ksoftirqd/309:33:353
445346995826,27cyclictest505482-21ssh11:10:000
44535499573,16cyclictest505041-21ssh11:08:593
445354995715,7cyclictest37-21ksoftirqd/309:14:013
445354995713,9cyclictest37-21ksoftirqd/311:03:163
445354995712,13cyclictest37-21ksoftirqd/309:19:223
445354995712,11cyclictest37-21ksoftirqd/310:48:453
445352995735,16cyclictest3467602-21systemd-journal08:45:002
445352995735,16cyclictest3467602-21systemd-journal08:45:002
445348995739,13cyclictest1-21systemd07:15:021
445348995739,13cyclictest1-21systemd07:15:011
445346995735,18cyclictest16850irq/134-800000012:40:000
445346995727,15cyclictest518276-21ssh11:38:360
445354995612,10cyclictest37-21ksoftirqd/309:36:563
445354995611,9cyclictest37-21ksoftirqd/308:40:003
445354995611,8cyclictest37-21ksoftirqd/310:32:493
445354995611,10cyclictest37-21ksoftirqd/307:40:013
445354995611,10cyclictest37-21ksoftirqd/307:40:013
445354995610,9cyclictest37-21ksoftirqd/312:00:543
445354995610,8cyclictest37-21ksoftirqd/311:18:053
445348995638,13cyclictest3467602-21systemd-journal08:10:011
445348995638,13cyclictest3467602-21systemd-journal08:10:001
445346995626,13cyclictest492659-21ssh10:40:550
445346995614,38cyclictest12-21ksoftirqd/012:01:150
44535499559,11cyclictest37-21ksoftirqd/312:15:113
445354995511,10cyclictest37-21ksoftirqd/311:30:003
445354995510,10cyclictest37-21ksoftirqd/309:22:433
445352995547,5cyclictest501007-21kworker/2:012:00:022
445352995546,5cyclictest439022-21kworker/2:108:40:002
445352995529,8cyclictest433666-21kworker/2:209:13:032
445346995511,6cyclictest131rcu_preempt09:17:570
44535499547,11cyclictest37-21ksoftirqd/308:20:023
44535499547,11cyclictest37-21ksoftirqd/308:20:013
44535499547,10cyclictest37-21ksoftirqd/311:11:313
445354995411,8cyclictest37-21ksoftirqd/310:59:353
445354995411,8cyclictest37-21ksoftirqd/308:45:013
445354995411,8cyclictest37-21ksoftirqd/308:45:003
445352995433,17cyclictest30-21ksoftirqd/208:30:002
44534699548,5cyclictest131rcu_preempt12:33:350
445346995423,28cyclictest12-21ksoftirqd/009:33:590
445354995310,9cyclictest37-21ksoftirqd/311:41:283
445352995345,5cyclictest458375-21kworker/2:309:32:372
445346995327,12cyclictest524635-21ssh11:52:280
44534699532,14cyclictest3467602-21systemd-journal09:15:000
44535499528,8cyclictest37-21ksoftirqd/311:31:563
44535499527,10cyclictest37-21ksoftirqd/311:35:423
445354995210,10cyclictest37-21ksoftirqd/309:10:023
445348995235,12cyclictest1-21systemd10:20:011
445346995244,4cyclictest16950irq/224-800000007:20:000
44534699522,45cyclictest467380-21ssh09:45:000
445346995222,14cyclictest460578-21ssh09:29:490
44535499519,7cyclictest37-21ksoftirqd/311:57:523
445354995113,8cyclictest37-21ksoftirqd/309:53:123
445352995143,5cyclictest501007-21kworker/2:012:28:022
445352995143,5cyclictest501007-21kworker/2:011:12:342
445352995143,5cyclictest433666-21kworker/2:209:42:442
445352995142,6cyclictest456478-21kworker/2:009:24:082
445352995142,5cyclictest501007-21kworker/2:012:00:152
445352995142,5cyclictest462890-21kworker/2:110:20:022
445352995138,8cyclictest447849-21cron07:59:592
445346995144,4cyclictest16950irq/224-800000010:14:590
445346995144,4cyclictest16950irq/224-800000008:40:000
44534699513,11cyclictest12-21ksoftirqd/008:05:010
445346995124,12cyclictest522852-21ssh11:48:370
445346995124,12cyclictest497343-21ssh10:51:320
44535499509,10cyclictest37-21ksoftirqd/308:25:003
44535499507,9cyclictest37-21ksoftirqd/310:50:173
44535499506,9cyclictest37-21ksoftirqd/310:20:443
44535499502,11cyclictest37-21ksoftirqd/312:20:143
445354995017,7cyclictest37-21ksoftirqd/310:06:413
445354995017,7cyclictest37-21ksoftirqd/310:06:403
445354995013,6cyclictest37-21ksoftirqd/310:18:353
445354995011,7cyclictest37-21ksoftirqd/311:22:223
445352995042,5cyclictest439022-21kworker/2:108:35:022
445352995042,5cyclictest439022-21kworker/2:108:35:012
445352995029,15cyclictest3467602-21systemd-journal09:40:012
445346995025,8cyclictest451510-21diskmemload11:15:010
445346995024,21cyclictest12-21ksoftirqd/010:20:000
445346995020,8cyclictest12-21ksoftirqd/008:45:020
445346995020,8cyclictest12-21ksoftirqd/008:45:010
445346995017,30cyclictest12-21ksoftirqd/011:57:080
44535499498,9cyclictest37-21ksoftirqd/311:45:343
44535499496,8cyclictest37-21ksoftirqd/309:46:563
445354994931,13cyclictest1-21systemd07:25:023
445354994931,13cyclictest1-21systemd07:25:013
445354994910,9cyclictest37-21ksoftirqd/308:00:003
445352994942,4cyclictest543671-21kworker/2:312:36:362
445352994941,5cyclictest462890-21kworker/2:111:39:272
445352994941,5cyclictest462890-21kworker/2:110:48:342
445352994941,5cyclictest462890-21kworker/2:110:33:162
445352994940,6cyclictest501007-21kworker/2:012:24:452
445352994940,6cyclictest433666-21kworker/2:210:10:522
445352994940,5cyclictest483146-21kworker/2:310:39:342
445348994934,11cyclictest3467602-21systemd-journal08:12:381
445348994932,13cyclictest275-21dbus-daemon12:05:011
445348994912,6cyclictest131rcu_preempt07:45:001
445346994928,16cyclictest500935-21ssh10:59:590
44534699491,5cyclictest0-21swapper/008:15:020
44534699491,44cyclictest509243-21ssh11:18:180
44534699491,43cyclictest476369-21ssh10:05:010
445346994910,6cyclictest131rcu_preempt11:41:170
44535499484,7cyclictest37-21ksoftirqd/307:35:013
44535499483,13cyclictest472249-21ssh09:55:253
445354994812,7cyclictest0-21swapper/307:20:003
445352994840,5cyclictest501007-21kworker/2:011:16:152
445352994840,5cyclictest462890-21kworker/2:112:15:512
445352994840,5cyclictest462890-21kworker/2:110:25:402
445352994840,5cyclictest462890-21kworker/2:109:51:272
445352994840,5cyclictest433666-21kworker/2:209:56:142
445352994838,6cyclictest462890-21kworker/2:111:08:022
445348994829,15cyclictest23-21ksoftirqd/109:25:021
445348994822,6cyclictest221rcuc/108:25:011
445346994841,4cyclictest16850irq/134-800000012:20:000
445346994840,5cyclictest16850irq/134-800000010:00:010
445346994819,21cyclictest151rcuc/011:03:410
445346994815,8cyclictest12-21ksoftirqd/009:53:300
445346994813,3cyclictest12-21ksoftirqd/010:30:000
445346994812,11cyclictest12-21ksoftirqd/010:09:420
445346994812,11cyclictest12-21ksoftirqd/010:09:410
445352994739,5cyclictest501007-21kworker/2:011:26:362
445352994739,5cyclictest462890-21kworker/2:111:30:402
445352994739,5cyclictest462890-21kworker/2:110:56:592
445352994737,6cyclictest433666-21kworker/2:207:55:022
445352994737,6cyclictest433666-21kworker/2:207:55:012
44534899473,4cyclictest131rcu_preempt11:05:001
44534699477,8cyclictest12-21ksoftirqd/008:30:000
445346994721,12cyclictest488541-21ssh10:31:480
445346994714,8cyclictest12-21ksoftirqd/011:30:200
44535499467,7cyclictest37-21ksoftirqd/309:05:023
445354994631,10cyclictest450989-21run-parts09:00:013
445354994612,5cyclictest0-21swapper/308:55:003
445352994638,5cyclictest462890-21kworker/2:111:49:402
445352994638,5cyclictest462890-21kworker/2:111:00:332
445352994633,8cyclictest309-21in:imuxsock08:20:022
445352994633,8cyclictest309-21in:imuxsock08:20:012
445352994629,12cyclictest456071-21cat09:20:012
445348994633,10cyclictest458827-21sshd09:25:401
445348994633,10cyclictest23-21ksoftirqd/110:39:231
445348994630,11cyclictest1-21systemd10:15:011
445348994628,13cyclictest275-21dbus-daemon07:55:011
445348994628,13cyclictest275-21dbus-daemon07:55:011
44534699465,36cyclictest0-21swapper/010:20:490
445346994636,6cyclictest12-21ksoftirqd/008:25:030
44534699462,39cyclictest510275-21ssh11:20:180
445346994616,5cyclictest131rcu_preempt08:35:010
445346994616,5cyclictest131rcu_preempt08:35:000
44535499456,9cyclictest37-21ksoftirqd/309:30:033
44535499456,9cyclictest37-21ksoftirqd/307:45:033
445352994538,4cyclictest462890-21kworker/2:111:40:482
445352994537,5cyclictest483146-21kworker/2:310:44:532
445352994537,5cyclictest462890-21kworker/2:110:51:052
445352994536,6cyclictest501007-21kworker/2:011:21:052
445352994536,5cyclictest462890-21kworker/2:111:54:292
445352994531,10cyclictest1-21systemd07:35:002
445348994536,5cyclictest451509-21sh09:10:001
44534899453,3cyclictest131rcu_preempt11:09:091
445348994529,12cyclictest275-21dbus-daemon07:59:591
445348994529,11cyclictest309-21in:imuxsock12:40:011
445348994527,13cyclictest275-21dbus-daemon10:25:001
445346994526,13cyclictest275-21dbus-daemon07:45:000
445346994512,9cyclictest12-21ksoftirqd/012:20:530
445346994512,9cyclictest12-21ksoftirqd/009:38:560
445352994436,5cyclictest433666-21kworker/2:210:05:262
445352994436,5cyclictest433666-21kworker/2:210:05:262
445352994434,6cyclictest446829-21sendmail07:40:012
445352994434,6cyclictest446829-21sendmail07:40:002
445352994412,6cyclictest131rcu_preempt08:25:002
445348994426,14cyclictest275-21dbus-daemon11:25:011
445348994414,5cyclictest0-21swapper/111:44:271
445346994428,12cyclictest275-21dbus-daemon08:00:000
445346994427,14cyclictest496016-21ssh10:48:480
445352994335,5cyclictest462890-21kworker/2:112:14:252
445352994334,6cyclictest501007-21kworker/2:012:32:512
445348994329,10cyclictest275-21dbus-daemon09:40:021
445346994331,9cyclictest12-21ksoftirqd/007:40:030
445346994331,9cyclictest12-21ksoftirqd/007:40:020
445346994330,9cyclictest451256-21modprobe09:05:000
445346994326,12cyclictest275-21dbus-daemon07:35:010
445354994224,14cyclictest447585-21sshd07:52:403
445354994224,14cyclictest447585-21sshd07:52:393
445354994223,15cyclictest450440-21sshd08:45:393
445354994210,9cyclictest0-21swapper/308:35:023
445354994210,9cyclictest0-21swapper/308:35:013
445352994234,5cyclictest433666-21kworker/2:209:27:172
445352994232,6cyclictest433666-21kworker/2:208:00:382
445348994232,6cyclictest500953-21sendmail10:59:591
445348994225,13cyclictest275-21dbus-daemon11:15:001
445348994224,14cyclictest275-21dbus-daemon10:50:001
44534899421,6cyclictest131rcu_preempt07:35:001
445346994229,9cyclictest12-21ksoftirqd/009:10:020
445346994228,11cyclictest12-21ksoftirqd/008:20:000
445346994228,11cyclictest12-21ksoftirqd/008:20:000
445346994228,10cyclictest12-21ksoftirqd/008:55:020
44535499417,6cyclictest445440-21latency_hist07:10:023
445354994126,11cyclictest449433-21latency_hist08:30:023
445352994131,7cyclictest1-21systemd07:45:002
445352994125,11cyclictest30-21ksoftirqd/209:00:002
445348994132,7cyclictest543548-21ssh12:34:381
445348994124,12cyclictest275-21dbus-daemon08:25:491
445348994120,15cyclictest275-21dbus-daemon08:05:001
445348994116,5cyclictest23-21ksoftirqd/112:07:311
445346994127,10cyclictest445604-21sed07:15:030
445346994127,10cyclictest445604-21sed07:15:020
445346994113,5cyclictest131rcu_preempt07:25:020
445346994113,5cyclictest131rcu_preempt07:25:010
445354994018,15cyclictest275-21dbus-daemon08:15:003
445352994032,5cyclictest433666-21kworker/2:210:00:112
445348994030,6cyclictest450712-21cron08:55:001
445348994028,9cyclictest1-21systemd10:30:021
445348994027,10cyclictest3467602-21systemd-journal11:46:031
445348994027,10cyclictest23-21ksoftirqd/112:14:221
44534899402,4cyclictest131rcu_preempt12:27:131
445348994024,11cyclictest1-21systemd09:50:011
445348994022,15cyclictest23-21ksoftirqd/110:31:221
445348994020,16cyclictest275-21dbus-daemon11:20:001
445348994016,8cyclictest23-21ksoftirqd/109:33:001
44534899401,3cyclictest131rcu_preempt11:57:051
445346994034,3cyclictest16950irq/224-800000008:05:520
445346994034,3cyclictest16950irq/224-800000008:05:520
445346994024,12cyclictest446429-21sed07:30:020
445346994016,6cyclictest131rcu_preempt07:50:020
445346994016,6cyclictest131rcu_preempt07:50:020
445354993926,9cyclictest448437-21sed08:10:023
445354993926,9cyclictest448437-21sed08:10:013
445354993926,9cyclictest275-21dbus-daemon07:15:013
445354993926,9cyclictest275-21dbus-daemon07:15:013
445352993931,5cyclictest433666-21kworker/2:209:48:202
445352993923,12cyclictest445787-21cron07:20:012
44535299391,4cyclictest131rcu_preempt09:10:002
44534899397,6cyclictest131rcu_preempt09:55:001
445348993927,9cyclictest23-21ksoftirqd/108:45:031
445348993927,9cyclictest23-21ksoftirqd/108:45:021
445348993926,11cyclictest23-21ksoftirqd/111:40:011
445348993925,10cyclictest309-21in:imuxsock07:38:391
445348993925,10cyclictest309-21in:imuxsock07:38:391
44534899392,3cyclictest131rcu_preempt11:34:091
445346993923,12cyclictest12-21ksoftirqd/007:50:000
445352993820,14cyclictest3467602-21systemd-journal07:25:002
445352993820,14cyclictest3467602-21systemd-journal07:25:002
445348993828,7cyclictest23-21ksoftirqd/107:20:031
445348993828,7cyclictest23-21ksoftirqd/107:20:021
445348993827,9cyclictest23-21ksoftirqd/112:17:211
445346993826,8cyclictest12-21ksoftirqd/009:00:000
445352993730,5cyclictest439022-21kworker/2:107:12:382
445352993730,5cyclictest439022-21kworker/2:107:12:382
445352993722,10cyclictest1-21systemd08:55:002
44534899373,3cyclictest131rcu_preempt12:22:251
445348993729,5cyclictest525439-21sh11:54:271
445348993722,11cyclictest23-21ksoftirqd/108:39:591
44534899372,18cyclictest0-21swapper/110:05:101
44534899372,18cyclictest0-21swapper/110:05:101
44534899371,3cyclictest131rcu_preempt09:13:131
44534699372,6cyclictest131rcu_preempt08:50:010
445352993627,6cyclictest448632-21cron08:14:592
44535299362,6cyclictest364-21snmpd08:06:402
44535299362,6cyclictest364-21snmpd08:06:392
445348993625,8cyclictest23-21ksoftirqd/108:50:011
445348993624,10cyclictest514451-21latency_hist11:30:021
445348993624,10cyclictest23-21ksoftirqd/109:56:401
445348993623,10cyclictest23-21ksoftirqd/110:42:281
445348993621,11cyclictest450965-21cron09:00:011
44534899361,4cyclictest131rcu_preempt09:41:561
44534899361,18cyclictest0-21swapper/109:17:111
44535499351,5cyclictest0-21swapper/307:47:403
44535299359,4cyclictest346771850lldpd08:48:472
445348993525,6cyclictest309-21in:imuxsock09:05:001
445348993525,5cyclictest449866-21latency_hist08:35:031
445348993525,5cyclictest449866-21latency_hist08:35:021
445348993522,10cyclictest23-21ksoftirqd/107:10:021
445348993518,4cyclictest23-21ksoftirqd/107:50:021
44534899350,6cyclictest150-21pr/ttyS208:18:321
44534899350,6cyclictest150-21pr/ttyS208:18:311
44534699353,5cyclictest131rcu_preempt07:10:020
44535299346,6cyclictest131rcu_preempt09:05:012
44534899343,3cyclictest131rcu_preempt07:20:001
445348993321,9cyclictest23-21ksoftirqd/107:30:011
445348993316,7cyclictest131rcu_preempt10:50:111
445352993218,11cyclictest3467602-21systemd-journal07:50:002
44535299311,4cyclictest131rcu_preempt07:30:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional