You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-16 - 08:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Sep 16, 2026 00:45:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
283912999726,15cyclictest296306-21ssh21:22:470
28391299921,37cyclictest380443-21ssh00:22:150
283912999125,14cyclictest331466-21ssh22:37:400
283912999086,2cyclictest0-21swapper/021:40:010
283912998426,13cyclictest316565-21ssh22:05:450
283912998328,11cyclictest305465-21ssh21:42:100
28391299821,40cyclictest364975-21ssh23:49:260
283912998128,16cyclictest360002-21ssh23:38:450
283912998013,39cyclictest302085-21ssh21:35:030
283912997927,12cyclictest369512-21ssh23:59:010
283912997916,57cyclictest16850irq/134-800000022:17:380
283912997827,11cyclictest333910-21ssh22:42:490
283912997732,5cyclictest372266-21sh00:05:010
283912997726,14cyclictest373574-21ssh00:07:460
28391299771,36cyclictest336441-21ssh22:48:140
283912997529,13cyclictest350985-21ssh23:19:340
283912997427,13cyclictest351603-21ssh23:20:180
283912997332,10cyclictest16950irq/224-800000023:30:020
283912997327,11cyclictest379062-21ssh00:19:430
28391299732,39cyclictest376494-21ssh00:14:070
28391299732,39cyclictest376494-21ssh00:14:070
283912997310,31cyclictest16850irq/134-800000021:28:200
283912997230,10cyclictest16950irq/224-800000023:09:590
283912997226,11cyclictest320488-21ssh22:14:330
283912997210,30cyclictest16850irq/134-800000000:30:490
283912997210,30cyclictest16850irq/134-800000000:30:480
28391599711,3cyclictest131rcu_preempt19:15:031
28391599711,3cyclictest131rcu_preempt19:15:021
283912997127,11cyclictest314992-21ssh22:02:310
28391299711,37cyclictest348777-21ssh23:14:520
283912997026,13cyclictest356120-21ssh23:30:060
283912996929,6cyclictest290106-21munin-plugin-st21:09:590
283912996916,48cyclictest16850irq/134-800000021:53:270
283912996914,50cyclictest16850irq/134-800000023:51:270
28391299682,13cyclictest313742-21modprobe22:00:000
283912996722,33cyclictest12-21ksoftirqd/022:26:380
283912996713,21cyclictest16850irq/134-800000023:03:030
283912996616,45cyclictest16850irq/134-800000022:33:080
283912996615,46cyclictest16850irq/134-800000000:27:470
283915996520,13cyclictest23-21ksoftirqd/122:30:001
283912996512,29cyclictest16850irq/134-800000023:40:380
283912996425,13cyclictest294514-21ssh21:19:180
283912996414,45cyclictest16850irq/134-800000022:57:160
28391599639,33cyclictest346698-21ssh23:10:021
28392499629,31cyclictest0-21swapper/322:40:023
28391599628,29cyclictest304330-21modprobe21:40:001
283912996215,43cyclictest16850irq/134-800000000:37:570
283912996210,31cyclictest16850irq/134-800000021:14:000
283919996114,32cyclictest0-21swapper/223:10:012
283915995943,14cyclictest221rcuc/120:40:011
28391299597,31cyclictest16850irq/134-800000022:23:450
283915995724,12cyclictest23-21ksoftirqd/123:43:421
28391599572,10cyclictest23-21ksoftirqd/100:00:171
283919995648,5cyclictest285717-21kworker/2:100:10:012
283919995636,7cyclictest323150-21kworker/2:323:30:012
283919995546,6cyclictest285717-21kworker/2:121:55:012
283919995541,9cyclictest283908-21cyclictest19:10:012
283919995537,13cyclictest275-21dbus-daemon19:20:002
28391599558,5cyclictest131rcu_preempt22:24:071
283915995540,8cyclictest285691-21munin-plugin-st19:45:011
283915995515,6cyclictest131rcu_preempt21:10:011
283912995527,13cyclictest308693-21ssh21:49:100
283919995344,6cyclictest285717-21kworker/2:122:17:502
283919995343,6cyclictest285717-21kworker/2:122:01:322
283915995326,8cyclictest23-21ksoftirqd/120:35:021
283915995326,8cyclictest23-21ksoftirqd/120:35:021
283912995326,19cyclictest0-21swapper/022:53:410
283912995326,19cyclictest0-21swapper/022:53:410
283912995318,30cyclictest278040-21kworker/0:120:05:030
283919995244,5cyclictest323150-21kworker/2:323:17:092
28391599529,5cyclictest131rcu_preempt20:30:001
28391599529,5cyclictest131rcu_preempt00:05:451
28391599527,6cyclictest131rcu_preempt23:29:591
283915995234,13cyclictest1-21systemd22:40:011
28392499518,9cyclictest37-21ksoftirqd/322:10:013
283919995143,5cyclictest285717-21kworker/2:122:07:382
283919995143,5cyclictest285717-21kworker/2:121:15:272
283919995143,5cyclictest285717-21kworker/2:100:03:462
283919995141,6cyclictest286538-21kworker/2:220:15:012
283919995111,9cyclictest131rcu_preempt21:31:372
283915995133,12cyclictest275-21dbus-daemon22:50:011
283915995129,17cyclictest23-21ksoftirqd/119:20:021
283915995110,5cyclictest131rcu_preempt23:40:021
28392499508,7cyclictest37-21ksoftirqd/323:20:013
283919995042,5cyclictest323150-21kworker/2:322:49:452
283919995042,5cyclictest285717-21kworker/2:121:24:022
283919995041,6cyclictest286538-21kworker/2:221:36:562
283915995037,10cyclictest23-21ksoftirqd/100:19:441
283915995014,6cyclictest131rcu_preempt00:30:011
28391599501,16cyclictest0-21swapper/123:30:311
28391599501,16cyclictest0-21swapper/122:57:261
283919994942,4cyclictest323150-21kworker/2:323:31:062
283919994941,5cyclictest323150-21kworker/2:323:23:322
283919994941,5cyclictest285717-21kworker/2:121:27:072
283919994940,6cyclictest323150-21kworker/2:322:44:162
283919994940,6cyclictest285717-21kworker/2:121:46:042
283919994940,6cyclictest285717-21kworker/2:100:29:372
283919994940,5cyclictest323150-21kworker/2:322:29:212
28391599499,4cyclictest131rcu_preempt23:20:021
283915994933,11cyclictest3467602-21systemd-journal23:05:001
28392499485,3cyclictest131rcu_preempt21:10:013
283919994840,5cyclictest323150-21kworker/2:323:36:522
283919994840,5cyclictest286538-21kworker/2:221:43:212
283919994840,5cyclictest285717-21kworker/2:123:51:142
283919994840,5cyclictest285717-21kworker/2:122:10:252
283919994840,5cyclictest285717-21kworker/2:121:57:352
283919994839,5cyclictest323150-21kworker/2:323:47:522
283919994839,5cyclictest323150-21kworker/2:323:05:022
283919994833,11cyclictest1-21systemd20:00:012
283915994834,8cyclictest1-21systemd21:00:011
283915994834,8cyclictest1-21systemd21:00:001
283915994821,7cyclictest23-21ksoftirqd/121:55:021
28391599481,8cyclictest3467602-21systemd-journal21:30:001
28391599481,5cyclictest131rcu_preempt00:15:021
28391599481,5cyclictest131rcu_preempt00:15:011
28391599481,16cyclictest0-21swapper/122:51:221
28391599481,16cyclictest0-21swapper/122:51:211
28391599481,14cyclictest0-21swapper/123:07:451
283912994811,6cyclictest131rcu_preempt19:15:020
283912994811,6cyclictest131rcu_preempt19:15:020
28392499473,6cyclictest131rcu_preempt21:38:093
283924994730,11cyclictest3467602-21systemd-journal22:30:013
28391999479,6cyclictest131rcu_preempt22:36:572
283919994739,5cyclictest285717-21kworker/2:100:30:162
283919994739,5cyclictest285717-21kworker/2:100:30:162
283919994739,5cyclictest285717-21kworker/2:100:15:502
283919994738,6cyclictest323150-21kworker/2:322:32:482
283919994737,7cyclictest285717-21kworker/2:120:40:022
283915994736,8cyclictest23-21ksoftirqd/121:47:521
283915994733,10cyclictest23-21ksoftirqd/122:02:191
283915994731,12cyclictest3467602-21systemd-journal21:41:011
283915994729,14cyclictest275-21dbus-daemon20:15:011
283915994715,6cyclictest131rcu_preempt21:59:591
283915994712,12cyclictest23-21ksoftirqd/120:20:011
283924994612,5cyclictest131rcu_preempt22:33:013
283919994639,4cyclictest323150-21kworker/2:323:11:302
283919994635,7cyclictest285717-21kworker/2:122:20:242
28391599469,6cyclictest131rcu_preempt22:09:051
28391599466,5cyclictest131rcu_preempt23:48:511
283915994615,11cyclictest23-21ksoftirqd/122:15:011
28391599461,4cyclictest131rcu_preempt20:55:001
28391599461,35cyclictest353711-21ssh23:25:031
28391599461,18cyclictest0-21swapper/123:59:421
28391299463,22cyclictest0-21swapper/019:54:590
28392499459,5cyclictest131rcu_preempt21:33:043
283924994531,10cyclictest1-21systemd21:15:013
28392499451,12cyclictest292796-21ssh21:15:163
283919994537,5cyclictest323150-21kworker/2:323:43:222
283919994537,5cyclictest286538-21kworker/2:220:08:312
283919994537,5cyclictest285717-21kworker/2:123:56:062
283919994537,5cyclictest285717-21kworker/2:121:11:172
28391599459,6cyclictest131rcu_preempt19:25:001
28391599458,6cyclictest131rcu_preempt22:20:011
28391599456,8cyclictest23-21ksoftirqd/119:50:341
28391599455,10cyclictest23-21ksoftirqd/100:22:091
283915994529,12cyclictest328704-21sshd22:31:381
283915994511,5cyclictest131rcu_preempt20:25:011
28392499447,5cyclictest131rcu_preempt22:13:263
28392499446,3cyclictest131rcu_preempt23:40:003
28392499443,5cyclictest131rcu_preempt00:40:003
283924994425,14cyclictest275-21dbus-daemon00:20:003
28392499441,6cyclictest0-21swapper/322:59:343
283924994412,5cyclictest131rcu_preempt21:55:023
283919994436,5cyclictest323150-21kworker/2:300:38:052
283919994435,5cyclictest285717-21kworker/2:100:11:042
283919994435,5cyclictest285717-21kworker/2:100:11:042
28391599447,6cyclictest131rcu_preempt00:32:411
28391599447,6cyclictest131rcu_preempt00:32:401
283915994424,15cyclictest275-21dbus-daemon21:25:011
283915994419,9cyclictest131rcu_preempt21:30:361
283912994432,8cyclictest287491-21cron20:20:010
28392499439,5cyclictest131rcu_preempt22:50:433
28392499439,5cyclictest131rcu_preempt22:50:433
28392499434,5cyclictest131rcu_preempt22:20:523
28392499433,3cyclictest131rcu_preempt21:57:343
283924994328,10cyclictest275-21dbus-daemon22:20:013
283924994326,12cyclictest275-21dbus-daemon20:20:013
28392499431,5cyclictest352574-21ssh23:22:453
283924994313,3cyclictest150-21pr/ttyS223:26:093
283924994311,5cyclictest131rcu_preempt21:22:443
28391999438,6cyclictest131rcu_preempt22:59:572
283919994335,5cyclictest285717-21kworker/2:100:24:352
283919994328,11cyclictest3467602-21systemd-journal21:05:002
28391599437,6cyclictest131rcu_preempt20:00:011
28391599435,6cyclictest131rcu_preempt20:50:001
28391599435,6cyclictest131rcu_preempt00:39:571
28391599431,13cyclictest0-21swapper/123:53:411
28392499429,5cyclictest131rcu_preempt22:47:183
28392499427,4cyclictest131rcu_preempt23:41:443
28392499426,5cyclictest131rcu_preempt23:46:103
28392499424,6cyclictest131rcu_preempt20:55:023
28392499424,5cyclictest131rcu_preempt23:12:203
283924994233,6cyclictest289835-21cron21:05:003
283924994224,13cyclictest275-21dbus-daemon20:10:013
283924994223,15cyclictest275-21dbus-daemon19:55:013
28392499421,8cyclictest345163-21ssh23:06:523
28392499421,5cyclictest381381-21modprobe00:24:323
283919994234,5cyclictest323150-21kworker/2:322:50:252
283919994234,5cyclictest323150-21kworker/2:322:50:242
283919994228,9cyclictest3467602-21systemd-journal19:55:002
283919994228,10cyclictest275-21dbus-daemon19:30:012
283919994217,6cyclictest131rcu_preempt20:20:012
28391599426,6cyclictest131rcu_preempt20:10:011
283915994234,5cyclictest283920-21latency_hist19:10:021
283915994227,11cyclictest3467602-21systemd-journal20:40:331
283915994211,6cyclictest131rcu_preempt19:40:031
283915994211,6cyclictest131rcu_preempt19:30:011
28391599421,16cyclictest0-21swapper/121:05:001
283912994236,3cyclictest16950irq/224-800000019:50:020
283912994236,3cyclictest16950irq/224-800000019:50:010
28392499419,5cyclictest131rcu_preempt22:04:203
28392499419,5cyclictest131rcu_preempt00:07:063
28392499417,5cyclictest131rcu_preempt21:26:003
28392499414,6cyclictest131rcu_preempt00:00:053
28392499412,10cyclictest366895-21ssh23:53:253
283919994132,6cyclictest309-21in:imuxsock20:50:012
283919994122,13cyclictest275-21dbus-daemon20:30:012
28391599419,6cyclictest131rcu_preempt20:05:011
28391599415,5cyclictest131rcu_preempt21:15:031
28391599411,6cyclictest0-21swapper/122:40:511
28391599411,5cyclictest0-21swapper/121:15:451
283912994129,9cyclictest1-21systemd19:30:020
283912994122,14cyclictest3467602-21systemd-journal20:50:000
283912994116,21cyclictest16850irq/134-800000020:29:330
28392499409,5cyclictest131rcu_preempt00:13:513
28392499409,5cyclictest131rcu_preempt00:13:503
28392499407,5cyclictest131rcu_preempt00:32:143
28392499407,5cyclictest131rcu_preempt00:32:133
28392499404,6cyclictest131rcu_preempt21:00:023
28392499404,6cyclictest131rcu_preempt21:00:013
28392499402,8cyclictest356761-21ssh23:31:373
283924994027,8cyclictest37-21ksoftirqd/319:15:033
283924994027,8cyclictest37-21ksoftirqd/319:15:023
28392499401,8cyclictest307218-21ssh21:45:483
28392499401,4cyclictest131rcu_preempt23:55:193
283919994023,12cyclictest275-21dbus-daemon21:00:012
283919994023,12cyclictest275-21dbus-daemon21:00:002
283912994035,3cyclictest16950irq/224-800000020:25:000
283912994029,8cyclictest12-21ksoftirqd/019:20:000
283912994023,13cyclictest285388-21sshd19:37:340
283912994021,13cyclictest3467602-21systemd-journal21:00:000
283912994021,13cyclictest3467602-21systemd-journal21:00:000
28392499397,5cyclictest131rcu_preempt20:46:033
28392499397,5cyclictest131rcu_preempt20:30:003
283924993930,6cyclictest285932-21cron19:50:003
283924993930,6cyclictest285932-21cron19:50:003
283924993926,10cyclictest275-21dbus-daemon19:30:023
28392499391,4cyclictest131rcu_preempt19:40:033
283919993931,5cyclictest285717-21kworker/2:121:08:332
283919993926,9cyclictest285662-21sshd19:41:172
28391299393,33cyclictest0-21swapper/020:40:000
283912993921,11cyclictest275-21dbus-daemon20:55:010
28392499386,5cyclictest131rcu_preempt19:22:323
28392499384,5cyclictest131rcu_preempt19:40:543
28392499383,3cyclictest131rcu_preempt20:40:003
28392499383,10cyclictest131rcu_preempt00:29:513
28392499382,4cyclictest131rcu_preempt21:45:013
283919993825,9cyclictest275-21dbus-daemon20:55:012
28391299384,3cyclictest131rcu_preempt19:45:010
283912993829,6cyclictest12-21ksoftirqd/019:25:020
283912993828,7cyclictest286636-21sed20:00:020
283912993824,10cyclictest12-21ksoftirqd/020:35:020
283912993824,10cyclictest12-21ksoftirqd/020:35:010
28392499374,3cyclictest131rcu_preempt23:01:193
28392499374,30cyclictest37-21ksoftirqd/320:45:023
283924993720,14cyclictest286825-21wc20:05:033
28392499371,5cyclictest332864-21ssh22:40:263
28391599376,5cyclictest131rcu_preempt19:35:011
283915993725,9cyclictest23-21ksoftirqd/119:48:111
283915993725,9cyclictest23-21ksoftirqd/119:48:101
283912993727,7cyclictest288977-21cat20:45:020
283912993718,15cyclictest289815-21sshd21:01:260
283912993715,18cyclictest16850irq/134-800000019:32:320
28392499367,5cyclictest131rcu_preempt20:33:033
28392499367,5cyclictest131rcu_preempt20:33:033
283924993624,9cyclictest286505-21cut20:00:013
283924993618,13cyclictest275-21dbus-daemon20:25:013
28392499361,3cyclictest131rcu_preempt20:15:003
28391999369,6cyclictest131rcu_preempt19:50:022
28391999369,6cyclictest131rcu_preempt19:50:012
283919993628,5cyclictest284437-21kworker/2:019:40:052
283912993623,9cyclictest1-21systemd20:10:000
28392499352,3cyclictest131rcu_preempt19:10:013
283924993522,11cyclictest37-21ksoftirqd/319:20:003
283919993526,7cyclictest30-21ksoftirqd/220:25:012
283919993525,8cyclictest30-21ksoftirqd/220:35:022
283919993525,8cyclictest30-21ksoftirqd/220:35:022
283919993521,10cyclictest3467602-21systemd-journal19:34:592
283919993520,11cyclictest1-21systemd20:05:022
28391299352,31cyclictest0-21swapper/020:15:000
283924993424,7cyclictest285156-21cron19:34:593
283919993425,6cyclictest284292-21cat19:15:032
283919993425,6cyclictest284292-21cat19:15:022
283919993424,7cyclictest30-21ksoftirqd/219:25:032
283919993423,8cyclictest30-21ksoftirqd/220:43:582
283912993125,4cyclictest283916-21latency_hist19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional