You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-16 - 06:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Tue Jun 16, 2026 00:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2593885999789,4cyclictest30-21ksoftirqd/222:45:022
2593876998675,3cyclictest0-21swapper/023:50:020
2593885998511,8cyclictest131rcu_preempt23:45:022
2593890998433,47cyclictest2624708-21cron22:05:003
2593890998433,47cyclictest2624708-21cron22:05:003
2593876998226,32cyclictest12-21ksoftirqd/000:22:140
2593881997353,2cyclictest0-21swapper/120:15:011
2593890997129,12cyclictest275-21dbus-daemon20:25:013
2593885997025,12cyclictest3467602-21systemd-journal00:32:152
2593885996933,14cyclictest1-21systemd21:10:002
2593876996625,11cyclictest12-21ksoftirqd/020:05:000
2593890996437,24cyclictest0-21swapper/319:10:023
2593876996435,5cyclictest131rcu_preempt00:05:360
2593876996432,7cyclictest12-21ksoftirqd/000:27:220
2593881996342,15cyclictest3467602-21systemd-journal00:28:131
259387699634,7cyclictest121ksoftirqd/023:18:350
259388199624,10cyclictest23-21ksoftirqd/122:05:011
259388199624,10cyclictest23-21ksoftirqd/122:05:011
2593876996241,17cyclictest16850irq/134-800000022:05:000
2593876996241,17cyclictest16850irq/134-800000022:05:000
259387699623,8cyclictest12-21ksoftirqd/021:19:430
2593890996124,14cyclictest275-21dbus-daemon00:00:033
2593890996124,14cyclictest275-21dbus-daemon00:00:023
2593885996131,9cyclictest30-21ksoftirqd/220:30:002
259387699618,5cyclictest131rcu_preempt00:11:120
259387699617,7cyclictest121ksoftirqd/023:08:090
259387699617,7cyclictest121ksoftirqd/023:08:090
259387699617,10cyclictest12-21ksoftirqd/019:55:000
259388599606,50cyclictest2624976-21kworker/u8:222:25:022
259387699607,5cyclictest131rcu_preempt21:36:120
259387699607,5cyclictest131rcu_preempt21:36:120
2593876996051,5cyclictest12-21ksoftirqd/021:50:010
2593876996051,5cyclictest12-21ksoftirqd/021:50:010
2593885995932,8cyclictest2656112-21kworker/2:023:50:382
2593885995848,5cyclictest2595659-21kworker/2:200:30:002
259388199586,5cyclictest131rcu_preempt00:00:011
259388199586,5cyclictest131rcu_preempt00:00:011
2593881995838,15cyclictest1-21systemd19:30:011
259387699588,5cyclictest151rcuc/020:00:010
259387699584,19cyclictest2609180-21wc21:30:020
259387699584,19cyclictest2609180-21wc21:30:020
2593876995818,11cyclictest12-21ksoftirqd/023:15:020
2593876995814,14cyclictest12-21ksoftirqd/021:45:010
2593890995726,11cyclictest37-21ksoftirqd/321:38:373
2593890995726,11cyclictest37-21ksoftirqd/321:38:373
259388199574,10cyclictest23-21ksoftirqd/123:00:011
259388199574,10cyclictest23-21ksoftirqd/123:00:011
259387699578,11cyclictest12-21ksoftirqd/021:51:150
259387699578,11cyclictest12-21ksoftirqd/021:51:150
2593876995728,8cyclictest12-21ksoftirqd/000:37:190
2593876995728,26cyclictest2651589-21ssh23:04:590
2593876995717,10cyclictest12-21ksoftirqd/021:35:000
2593881995615,9cyclictest131rcu_preempt23:45:271
259387699569,8cyclictest12-21ksoftirqd/021:00:010
2593876995639,13cyclictest1-21systemd21:15:010
2593876995627,14cyclictest2634571-21ssh22:26:460
2593876995611,16cyclictest131rcu_preempt23:21:560
2593890995516,7cyclictest131rcu_preempt20:40:003
2593885995532,7cyclictest2599289-21kworker/2:321:36:312
2593885995532,7cyclictest2599289-21kworker/2:321:36:312
2593881995524,12cyclictest23-21ksoftirqd/122:50:011
259387699554,13cyclictest12-21ksoftirqd/000:05:010
2593876995534,17cyclictest16850irq/134-800000023:31:030
2593876995528,7cyclictest12-21ksoftirqd/020:20:010
2593876995519,19cyclictest16850irq/134-800000023:40:470
2593876995514,10cyclictest12-21ksoftirqd/022:55:020
2593890995437,13cyclictest1-21systemd19:50:013
259388599546,8cyclictest30-21ksoftirqd/223:00:022
259388599546,8cyclictest30-21ksoftirqd/223:00:022
2593885995417,17cyclictest0-21swapper/222:51:492
259387699548,5cyclictest16850irq/134-800000022:50:010
259387699547,7cyclictest12-21ksoftirqd/022:41:310
259387699546,8cyclictest12-21ksoftirqd/022:40:010
2593876995446,4cyclictest16950irq/224-800000019:40:000
2593876995426,13cyclictest2606086-21ssh21:23:230
2593876995419,6cyclictest16850irq/134-800000023:52:440
2593876995419,6cyclictest16850irq/134-800000022:59:500
2593876995419,6cyclictest16850irq/134-800000022:59:500
259388199536,5cyclictest131rcu_preempt22:55:021
2593881995320,10cyclictest23-21ksoftirqd/120:35:011
2593881995315,6cyclictest131rcu_preempt23:15:011
259387699536,8cyclictest12-21ksoftirqd/022:12:150
259387699532,46cyclictest3467602-21systemd-journal20:25:010
2593876995317,7cyclictest16850irq/134-800000022:18:160
2593885995244,5cyclictest2656112-21kworker/2:000:35:062
2593885995243,5cyclictest2595659-21kworker/2:222:50:002
2593885995234,12cyclictest275-21dbus-daemon20:40:002
259388199526,5cyclictest131rcu_preempt23:25:441
259388199526,5cyclictest131rcu_preempt23:25:441
259388199524,7cyclictest23-21ksoftirqd/119:15:021
2593881995229,16cyclictest2595355-21cron19:40:001
259387699528,13cyclictest12-21ksoftirqd/022:33:100
259387699525,8cyclictest12-21ksoftirqd/023:27:070
259387699525,8cyclictest12-21ksoftirqd/023:27:070
259387699525,12cyclictest12-21ksoftirqd/019:25:000
2593876995228,9cyclictest2621182-21ssh21:56:530
2593876995224,12cyclictest2675043-21ssh23:56:470
2593876995224,12cyclictest2675043-21ssh23:56:470
2593876995218,17cyclictest16850irq/134-800000023:35:310
2593885995143,5cyclictest2595659-21kworker/2:200:09:422
2593885995143,5cyclictest2594373-21kworker/2:119:50:012
2593885995142,6cyclictest2656112-21kworker/2:023:47:182
2593885995142,6cyclictest2656112-21kworker/2:023:18:182
259388599511,3cyclictest131rcu_preempt22:05:012
259388599511,3cyclictest131rcu_preempt22:05:012
259388199515,5cyclictest131rcu_preempt22:12:041
2593881995133,8cyclictest23-21ksoftirqd/100:36:051
259388199512,4cyclictest131rcu_preempt21:20:281
2593881995119,10cyclictest23-21ksoftirqd/120:00:011
2593881995110,6cyclictest131rcu_preempt21:34:121
259387699518,9cyclictest12-21ksoftirqd/000:17:510
259387699513,11cyclictest12-21ksoftirqd/019:45:020
2593890995034,12cyclictest1-21systemd21:20:013
2593885995042,5cyclictest2656112-21kworker/2:023:27:462
2593885995042,5cyclictest2656112-21kworker/2:023:27:452
2593885995042,5cyclictest2595659-21kworker/2:222:38:582
2593885995041,6cyclictest2656112-21kworker/2:000:00:522
2593885995041,5cyclictest2595659-21kworker/2:200:12:552
2593885995039,8cyclictest2595659-21kworker/2:222:05:302
2593885995034,12cyclictest2617990-21systemd-run21:50:012
2593885995034,12cyclictest2617990-21systemd-run21:50:012
259388199509,7cyclictest23-21ksoftirqd/121:56:281
259388199508,13cyclictest23-21ksoftirqd/100:23:131
259388199503,7cyclictest23-21ksoftirqd/100:20:011
2593881995013,6cyclictest131rcu_preempt23:40:511
259388199501,23cyclictest0-21swapper/100:10:011
2593881995011,6cyclictest131rcu_preempt21:15:011
259387699505,6cyclictest12-21ksoftirqd/019:17:000
2593890994934,12cyclictest1-21systemd00:25:003
259388599496,8cyclictest30-21ksoftirqd/223:40:022
2593885994941,5cyclictest2599289-21kworker/2:321:25:062
2593885994941,5cyclictest2599289-21kworker/2:321:25:062
2593885994941,5cyclictest2595659-21kworker/2:200:23:522
2593885994940,6cyclictest2599289-21kworker/2:323:02:162
2593885994940,5cyclictest2595659-21kworker/2:222:32:302
2593885994934,10cyclictest3467602-21systemd-journal00:20:002
259388199494,5cyclictest131rcu_preempt21:50:011
259388199494,5cyclictest131rcu_preempt21:50:011
2593881994940,6cyclictest23-21ksoftirqd/122:23:421
2593881994929,14cyclictest3467602-21systemd-journal22:40:001
2593881994916,8cyclictest23-21ksoftirqd/121:55:011
2593881994916,8cyclictest23-21ksoftirqd/121:55:011
259387699497,6cyclictest12-21ksoftirqd/020:30:020
259387699495,11cyclictest12-21ksoftirqd/022:24:410
2593876994941,5cyclictest151rcuc/000:30:210
259387699492,9cyclictest2627036-21sed22:10:010
259387699492,10cyclictest12-21ksoftirqd/020:50:010
259389099484,5cyclictest131rcu_preempt22:00:003
2593885994841,4cyclictest2595659-21kworker/2:222:17:112
2593885994840,5cyclictest2599289-21kworker/2:321:40:162
2593885994840,5cyclictest2595659-21kworker/2:222:27:552
2593885994839,6cyclictest2595659-21kworker/2:221:57:462
2593885994838,6cyclictest2599289-21kworker/2:321:14:522
259388199487,5cyclictest131rcu_preempt23:17:281
2593881994815,7cyclictest23-21ksoftirqd/121:26:161
2593881994815,7cyclictest23-21ksoftirqd/121:26:161
259388199481,26cyclictest0-21swapper/121:05:011
259387699488,7cyclictest12-21ksoftirqd/019:10:020
259387699486,8cyclictest2598007-21latency_hist20:30:010
2593876994832,11cyclictest3467602-21systemd-journal21:10:000
259387699481,14cyclictest364-21snmpd19:15:010
2593890994737,6cyclictest2667357-21sh23:40:003
259389099473,4cyclictest131rcu_preempt22:55:023
2593890994731,11cyclictest3467602-21systemd-journal22:45:013
2593885994739,5cyclictest2656112-21kworker/2:023:57:592
2593885994739,5cyclictest2656112-21kworker/2:023:57:582
2593885994739,5cyclictest2599289-21kworker/2:321:15:222
2593885994730,11cyclictest3467602-21systemd-journal20:00:012
2593885994710,9cyclictest131rcu_preempt21:33:312
259388199479,5cyclictest131rcu_preempt22:45:021
259388199476,6cyclictest131rcu_preempt00:05:001
259388199476,10cyclictest23-21ksoftirqd/121:10:011
259388199475,7cyclictest23-21ksoftirqd/120:50:011
259388199475,10cyclictest23-21ksoftirqd/121:40:131
259388199474,7cyclictest23-21ksoftirqd/121:17:551
259388199474,5cyclictest131rcu_preempt23:37:461
2593881994713,5cyclictest131rcu_preempt00:32:111
2593881994710,6cyclictest131rcu_preempt19:25:001
2593890994627,15cyclictest275-21dbus-daemon00:15:023
2593885994638,5cyclictest2599289-21kworker/2:321:23:382
2593885994638,5cyclictest2595659-21kworker/2:223:30:032
2593885994638,5cyclictest2594373-21kworker/2:120:51:122
259388199469,8cyclictest131rcu_preempt23:51:211
2593881994632,10cyclictest3467602-21systemd-journal19:35:001
259388199461,7cyclictest2653868-21ssh23:10:011
259388199461,7cyclictest2653868-21ssh23:10:011
259388199461,7cyclictest150-21pr/ttyS221:40:021
259388199461,7cyclictest150-21pr/ttyS221:40:021
2593881994616,19cyclictest23-21ksoftirqd/121:00:021
2593881994611,5cyclictest131rcu_preempt19:45:021
259388199461,12cyclictest23-21ksoftirqd/123:35:011
2593876994611,31cyclictest12-21ksoftirqd/021:01:120
259389099451,3cyclictest131rcu_preempt21:45:013
2593890994510,6cyclictest131rcu_preempt00:30:023
2593885994537,5cyclictest2595659-21kworker/2:223:12:132
2593885994537,5cyclictest2595659-21kworker/2:222:12:532
2593885994537,5cyclictest2595659-21kworker/2:221:54:182
2593885994537,5cyclictest2595659-21kworker/2:221:54:182
2593885994532,9cyclictest2594859-21cron19:30:012
2593885994511,6cyclictest131rcu_preempt23:24:402
259388199459,5cyclictest131rcu_preempt22:07:141
2593881994534,8cyclictest23-21ksoftirqd/122:31:581
2593881994533,8cyclictest23-21ksoftirqd/100:13:591
2593881994515,11cyclictest23-21ksoftirqd/119:10:021
259387699455,9cyclictest12-21ksoftirqd/020:45:020
2593876994517,5cyclictest131rcu_preempt20:05:110
259387699451,10cyclictest12-21ksoftirqd/019:35:010
259389099443,4cyclictest131rcu_preempt22:59:033
259389099443,4cyclictest131rcu_preempt22:59:033
2593890994429,12cyclictest37-21ksoftirqd/300:36:083
2593890994428,12cyclictest1-21systemd00:10:013
2593890994428,11cyclictest37-21ksoftirqd/319:15:013
2593890994427,13cyclictest2632653-21sshd22:22:293
2593885994438,3cyclictest2594373-21kworker/2:120:15:012
2593885994435,6cyclictest2599289-21kworker/2:323:10:022
2593885994435,6cyclictest2599289-21kworker/2:323:10:022
2593885994425,12cyclictest275-21dbus-daemon20:05:002
259388199443,9cyclictest23-21ksoftirqd/122:27:441
2593881994433,8cyclictest23-21ksoftirqd/122:16:531
2593881994433,8cyclictest23-21ksoftirqd/120:40:001
259387699448,9cyclictest0-21swapper/019:50:000
259389099432,3cyclictest131rcu_preempt22:05:333
259388599433,6cyclictest2595357-21cron19:39:592
2593881994318,5cyclictest131rcu_preempt20:05:001
259388199431,22cyclictest0-21swapper/119:20:011
259387699434,8cyclictest12-21ksoftirqd/020:10:030
259389099425,7cyclictest37-21ksoftirqd/323:31:563
259389099424,4cyclictest131rcu_preempt21:28:053
259389099424,4cyclictest131rcu_preempt21:28:053
2593890994228,10cyclictest2599527-21systemctl21:00:003
2593890994226,12cyclictest37-21ksoftirqd/322:39:583
259389099421,7cyclictest0-21swapper/323:08:473
259389099421,7cyclictest0-21swapper/323:08:463
259389099421,6cyclictest0-21swapper/323:26:523
259389099421,6cyclictest0-21swapper/323:26:523
259389099421,4cyclictest131rcu_preempt20:55:023
259389099421,4cyclictest131rcu_preempt00:00:273
259389099421,3cyclictest131rcu_preempt19:20:003
259388599425,5cyclictest131rcu_preempt19:24:262
2593885994232,6cyclictest2586101-21kworker/2:019:12:022
2593881994217,20cyclictest131rcu_preempt23:00:341
259388199421,10cyclictest23-21ksoftirqd/123:20:521
259387699422,36cyclictest2594996-21date19:30:020
259389099416,7cyclictest37-21ksoftirqd/323:10:163
259389099413,4cyclictest131rcu_preempt22:28:463
2593890994130,8cyclictest37-21ksoftirqd/319:25:003
2593890994125,11cyclictest3467602-21systemd-journal20:45:023
259389099412,4cyclictest131rcu_preempt22:48:013
259389099411,4cyclictest131rcu_preempt22:30:253
259388599413,5cyclictest131rcu_preempt20:58:492
259388199414,7cyclictest23-21ksoftirqd/120:42:161
2593881994128,10cyclictest23-21ksoftirqd/120:50:211
2593881994127,11cyclictest23-21ksoftirqd/120:25:021
259387699414,7cyclictest12-21ksoftirqd/020:40:010
259389099404,3cyclictest150-21pr/ttyS223:15:593
259389099403,9cyclictest311-21rs:main3
259389099403,3cyclictest131rcu_preempt21:35:023
259389099402,21cyclictest0-21swapper/300:18:433
259389099401,4cyclictest131rcu_preempt21:21:503
2593890994011,8cyclictest131rcu_preempt21:51:163
2593890994011,8cyclictest131rcu_preempt21:51:163
2593885994026,9cyclictest1-21systemd20:35:002
2593881994030,7cyclictest221rcuc/120:05:021
259389099393,6cyclictest131rcu_preempt20:00:003
2593890993926,11cyclictest37-21ksoftirqd/323:02:523
2593890993925,10cyclictest37-21ksoftirqd/321:10:013
2593890993925,10cyclictest275-21dbus-daemon21:15:013
2593890993924,12cyclictest37-21ksoftirqd/323:54:043
259389099391,4cyclictest131rcu_preempt22:20:013
259388199393,5cyclictest131rcu_preempt19:50:011
259387699394,10cyclictest12-21ksoftirqd/020:55:010
2593890993829,6cyclictest2669959-21sh23:45:263
2593890993825,9cyclictest37-21ksoftirqd/300:32:533
2593890993825,10cyclictest2597541-21cat20:20:023
2593890993822,11cyclictest1-21systemd20:05:013
259389099381,8cyclictest0-21swapper/323:23:013
2593885993831,4cyclictest2594373-21kworker/2:120:22:242
2593881993826,6cyclictest309-21in:imuxsock20:20:011
2593881993822,11cyclictest3467602-21systemd-journal20:30:001
259389099374,5cyclictest131rcu_preempt20:10:003
259389099374,4cyclictest131rcu_preempt19:35:003
259389099374,3cyclictest131rcu_preempt19:30:013
259389099372,4cyclictest131rcu_preempt21:48:453
259389099372,4cyclictest131rcu_preempt21:48:453
259389099372,3cyclictest131rcu_preempt19:55:013
259389099371,3cyclictest131rcu_preempt20:30:003
2593885993729,5cyclictest2597448-21sh20:20:002
259388599371,7cyclictest0-21swapper/220:10:022
2593881993728,6cyclictest23-21ksoftirqd/119:50:091
2593890993626,7cyclictest37-21ksoftirqd/320:45:013
2593890993620,11cyclictest275-21dbus-daemon19:40:013
259389099361,20cyclictest0-21swapper/322:12:043
259388599363,6cyclictest131rcu_preempt19:45:012
2593885993624,9cyclictest275-21dbus-daemon20:45:012
2593885993623,10cyclictest3467602-21systemd-journal21:02:122
259388599361,5cyclictest131rcu_preempt20:50:022
259389099351,3cyclictest131rcu_preempt21:02:123
259389099351,3cyclictest131rcu_preempt19:45:013
2593885993525,7cyclictest30-21ksoftirqd/219:55:022
2593885993521,10cyclictest30-21ksoftirqd/219:20:002
2593890993221,8cyclictest37-21ksoftirqd/320:33:133
2593890993220,9cyclictest37-21ksoftirqd/320:12:313
259388599322,4cyclictest131rcu_preempt19:35:012
2593885992921,5cyclictest2593870-21cyclictest19:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional