You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 22:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Mon Apr 20, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
185258321320,3sleep20-21swapper/208:20:002
185094921210,3sleep10-21swapper/108:15:031
182604521010,4sleep234-21ksoftirqd/207:05:132
18704752990,4sleep13056-21PK-Backend09:10:171
18268482800,7sleep00-21swapper/007:05:270
182702227639,10sleep30-21swapper/307:06:323
182725826243,14sleep10-21swapper/107:09:361
19415482430,4sleep33056-21PK-Backend12:30:193
18798392410,2sleep00-21swapper/009:35:220
18709942380,1sleep31870993-21timerandwakeup09:10:233
139099350,3rtkit-daemon1389-21rtkit-daemon12:16:571
1827423992914,13cyclictest1852482-21unin-run08:20:013
182741599275,5cyclictest3056-21PK-Backend08:54:591
182742399261,1cyclictest3056-21PK-Backend11:10:003
1827423992520,4cyclictest42-21ksoftirqd/308:50:013
1827423992520,4cyclictest42-21ksoftirqd/308:50:013
1827415992516,6cyclictest1848712-21cron08:10:001
182741599242,10cyclictest1873661-21tr09:20:131
1827413992419,3cyclictest1935084-21nm-dispatcher12:12:130
139099240,3rtkit-daemon1389-21rtkit-daemon10:53:380
139099240,3rtkit-daemon1389-21rtkit-daemon10:34:570
19401212230,5sleep20-21swapper/212:25:222
182742399239,4cyclictest42-21ksoftirqd/307:35:423
182742399230,2cyclictest3056-21PK-Backend12:25:003
182742399230,2cyclictest3056-21PK-Backend11:50:013
182742099238,3cyclictest141rcu_preempt10:40:002
1827420992319,3cyclictest1941488-21cut12:30:182
182742099230,21cyclictest1890668-21chrt10:07:592
139099230,4rtkit-daemon1389-21rtkit-daemon11:38:010
18439812220,4sleep01843987-21cpuspeed_turbos07:55:130
182742399226,3cyclictest141rcu_preempt11:55:173
1827423992217,3cyclictest18347252sleep307:29:223
1827423992215,4cyclictest1854190-21idleruntime-cro08:25:013
1827423992211,9cyclictest1857683-21unin-run08:35:013
1827423992210,6cyclictest141rcu_preempt07:54:393
1827420992215,6cyclictest1887385-21/usr/sbin/munin10:00:012
182741599224,6cyclictest1896615-21runrttasks10:21:161
182741599224,5cyclictest1905517-21runrttasks10:49:141
1827415992220,1cyclictest0-21swapper/111:53:261
1827415992220,1cyclictest0-21swapper/111:39:011
1827415992220,1cyclictest0-21swapper/111:30:431
1827415992220,1cyclictest0-21swapper/111:07:371
1827415992220,1cyclictest0-21swapper/107:37:491
1827415992219,2cyclictest3056-21PK-Backend12:04:291
1827415992219,2cyclictest3056-21PK-Backend09:08:001
1827415992219,2cyclictest3056-21PK-Backend08:46:511
1827415992219,2cyclictest3056-21PK-Backend08:46:501
1827415992218,3cyclictest1849854-21python308:10:201
182741599220,8cyclictest1929885-21taskset11:57:161
182741599220,5cyclictest26-21ksoftirqd/107:58:371
1827413992220,1cyclictest0-21swapper/009:47:110
1827413992220,1cyclictest0-21swapper/009:42:450
182741399220,21cyclictest2156-21gnome-shell08:33:080
182741399220,21cyclictest0-21swapper/009:05:540
139099220,3rtkit-daemon1389-21rtkit-daemon07:16:321
18869352210,10sleep013-21ksoftirqd/009:55:220
182742399213,3cyclictest141rcu_preempt09:01:343
1827423992119,1cyclictest3056-21PK-Backend11:50:543
1827423992119,1cyclictest0-21swapper/308:59:373
1827423992118,2cyclictest0-21swapper/309:39:323
1827423992117,3cyclictest42-21ksoftirqd/309:55:013
1827423992116,3cyclictest19036632sleep310:42:413
1827423992111,8cyclictest3056-21PK-Backend12:30:003
182742399211,17cyclictest1888506-21python310:00:193
182742399210,20cyclictest3056-21PK-Backend12:10:543
182742399210,20cyclictest3056-21PK-Backend11:34:033
182742399210,19cyclictest3056-21PK-Backend11:18:293
1827420992114,6cyclictest883-21dbus-daemon11:58:042
1827420992114,5cyclictest883-21dbus-daemon08:38:032
1827420992112,7cyclictest1854454-21apt08:25:122
182742099210,2cyclictest19124672sleep211:09:302
182742099210,19cyclictest3056-21PK-Backend11:40:052
182741599212,14cyclictest1840332-21taskset07:45:121
1827415992119,1cyclictest3056-21PK-Backend09:48:001
1827415992119,1cyclictest0-21swapper/112:25:311
1827415992119,1cyclictest0-21swapper/112:22:261
1827415992119,1cyclictest0-21swapper/111:13:051
1827415992119,1cyclictest0-21swapper/110:43:451
1827415992119,1cyclictest0-21swapper/110:10:461
1827415992119,1cyclictest0-21swapper/109:35:101
1827415992119,1cyclictest0-21swapper/109:29:571
1827415992118,2cyclictest251rcuc/109:53:551
1827415992114,5cyclictest883-21dbus-daemon07:15:011
1827415992114,5cyclictest1935334-21munin-run12:15:001
182741599210,6cyclictest26-21ksoftirqd/109:03:341
182741599210,6cyclictest26-21ksoftirqd/108:26:441
182741599210,20cyclictest3056-21PK-Backend08:36:091
182741599210,20cyclictest0-21swapper/111:03:591
182741599210,1cyclictest1831597-21apt07:20:121
1827413992119,1cyclictest3056-21PK-Backend08:35:250
1827413992119,1cyclictest0-21swapper/011:51:340
1827413992119,1cyclictest0-21swapper/011:15:050
1827413992119,1cyclictest0-21swapper/010:43:560
1827413992119,1cyclictest0-21swapper/008:52:560
1827413992119,1cyclictest0-21swapper/007:52:470
1827413992119,1cyclictest0-21swapper/007:10:550
182741399210,20cyclictest0-21swapper/010:55:170
182741399210,20cyclictest0-21swapper/010:46:460
182741399210,20cyclictest0-21swapper/008:55:460
182741399210,20cyclictest0-21swapper/007:22:530
182742399209,4cyclictest42-21ksoftirqd/311:14:363
182742399209,4cyclictest42-21ksoftirqd/309:05:363
182742399209,4cyclictest42-21ksoftirqd/308:07:373
182742399209,4cyclictest42-21ksoftirqd/308:00:403
182742399208,4cyclictest42-21ksoftirqd/309:31:423
182742399207,6cyclictest42-21ksoftirqd/309:49:273
182742399204,7cyclictest141rcu_preempt08:42:383
1827423992016,3cyclictest42-21ksoftirqd/310:10:003
1827423992015,2cyclictest1880718-21apt09:40:113
1827423992010,8cyclictest2277-21gvfsd-trash10:18:423
182742399200,7cyclictest3056-21PK-Backend07:30:173
182742399200,19cyclictest3056-21PK-Backend10:39:283
182742399200,19cyclictest0-21swapper/310:30:223
182742399200,19cyclictest0-21swapper/309:16:423
182742399200,18cyclictest3056-21PK-Backend11:01:313
1827420992018,1cyclictest0-21swapper/212:07:372
1827420992018,1cyclictest0-21swapper/212:03:322
1827420992018,1cyclictest0-21swapper/211:47:362
1827420992018,1cyclictest0-21swapper/208:40:192
182742099200,1cyclictest0-21swapper/207:26:172
182742099200,19cyclictest0-21swapper/211:52:302
182742099200,19cyclictest0-21swapper/211:39:042
182742099200,19cyclictest0-21swapper/211:21:282
182742099200,19cyclictest0-21swapper/211:10:322
182742099200,19cyclictest0-21swapper/211:02:282
182742099200,19cyclictest0-21swapper/210:50:322
182742099200,19cyclictest0-21swapper/210:28:142
182742099200,19cyclictest0-21swapper/210:16:172
182742099200,19cyclictest0-21swapper/210:04:222
182742099200,19cyclictest0-21swapper/209:46:512
182742099200,19cyclictest0-21swapper/209:42:242
182742099200,19cyclictest0-21swapper/209:24:272
182742099200,19cyclictest0-21swapper/208:34:452
182742099200,19cyclictest0-21swapper/207:44:292
182742099200,19cyclictest0-21swapper/207:12:182
182741599207,6cyclictest1917843-21taskset11:23:251
182741599204,3cyclictest1878616-21taskset09:33:561
1827415992018,1cyclictest3056-21PK-Backend12:35:591
1827415992018,1cyclictest3056-21PK-Backend12:35:581
1827415992018,1cyclictest3056-21PK-Backend10:01:541
1827415992018,1cyclictest0-21swapper/111:43:401
1827415992018,1cyclictest0-21swapper/107:44:321
1827415992018,1cyclictest0-21swapper/107:25:051
1827415992013,5cyclictest1835707-21needreboot07:30:171
1827415992011,7cyclictest1861121-21cron08:45:001
182741599200,6cyclictest26-21ksoftirqd/108:33:021
182741599200,1cyclictest0-21swapper/110:27:561
182741599200,19cyclictest3056-21PK-Backend12:06:321
182741599200,19cyclictest3056-21PK-Backend08:57:051
182741599200,19cyclictest0-21swapper/111:46:301
182741599200,19cyclictest0-21swapper/111:29:401
182741599200,19cyclictest0-21swapper/110:59:321
182741599200,19cyclictest0-21swapper/110:17:001
182741599200,19cyclictest0-21swapper/107:51:271
1827413992018,1cyclictest0-21swapper/012:04:250
1827413992018,1cyclictest0-21swapper/011:46:420
1827413992018,1cyclictest0-21swapper/011:33:290
1827413992018,1cyclictest0-21swapper/011:27:010
1827413992018,1cyclictest0-21swapper/010:39:160
1827413992018,1cyclictest0-21swapper/010:21:520
1827413992018,1cyclictest0-21swapper/010:16:240
1827413992018,1cyclictest0-21swapper/010:02:520
1827413992018,1cyclictest0-21swapper/008:40:460
1827413992018,1cyclictest0-21swapper/007:38:540
1827413992018,1cyclictest0-21swapper/007:16:240
1827413992016,3cyclictest1876716-21apt-get09:30:030
1827413992014,5cyclictest1911523-21interrupts11:05:170
182741399200,1cyclictest0-21swapper/011:20:330
182741399200,1cyclictest0-21swapper/008:17:340
182741399200,19cyclictest0-21swapper/011:01:320
182741399200,19cyclictest0-21swapper/010:07:190
182741399200,19cyclictest0-21swapper/009:04:520
139099200,3rtkit-daemon1389-21rtkit-daemon11:44:360
139099200,3rtkit-daemon1389-21rtkit-daemon09:20:300
139099200,3rtkit-daemon1389-21rtkit-daemon07:44:180
18529082190,3sleep13056-21PK-Backend08:20:111
182742399199,3cyclictest42-21ksoftirqd/307:22:153
182742399198,4cyclictest42-21ksoftirqd/311:20:343
182742399196,2cyclictest141rcu_preempt09:25:293
1827423991917,1cyclictest3056-21PK-Backend10:56:193
1827423991917,1cyclictest3056-21PK-Backend10:53:283
1827423991917,1cyclictest3056-21PK-Backend07:13:253
1827423991916,2cyclictest3056-21PK-Backend12:35:013
1827423991916,2cyclictest3056-21PK-Backend12:35:003
1827423991916,2cyclictest3056-21PK-Backend12:16:493
1827423991912,5cyclictest1874544-21cat09:20:233
182742399191,16cyclictest1843996-21cstates07:55:133
182742399190,9cyclictest42-21ksoftirqd/308:25:183
182742399190,6cyclictest3056-21PK-Backend07:45:143
182742399190,3cyclictest3056-21PK-Backend10:25:163
182742399190,2cyclictest3056-21PK-Backend10:20:203
182742399190,1cyclictest3056-21PK-Backend11:42:223
182742399190,1cyclictest3056-21PK-Backend07:17:513
182742399190,18cyclictest0-21swapper/312:07:503
182742399190,18cyclictest0-21swapper/312:04:453
182742399190,18cyclictest0-21swapper/310:14:003
182742399190,18cyclictest0-21swapper/309:57:523
182742399190,18cyclictest0-21swapper/308:35:243
182742399190,17cyclictest436-20systemd-journal08:50:133
1827420991917,1cyclictest0-21swapper/211:30:472
1827420991917,1cyclictest0-21swapper/210:47:402
1827420991917,1cyclictest0-21swapper/210:24:362
1827420991917,1cyclictest0-21swapper/210:10:492
1827420991917,1cyclictest0-21swapper/209:10:472
1827420991917,1cyclictest0-21swapper/209:06:552
1827420991917,1cyclictest0-21swapper/207:32:472
1827420991915,3cyclictest978-21snmpd07:22:382
1827420991913,4cyclictest1841790-21sh07:49:592
1827420991911,6cyclictest1843974-21expr07:55:132
182742099190,1cyclictest0-21swapper/210:58:502
182742099190,1cyclictest0-21swapper/209:01:152
182742099190,1cyclictest0-21swapper/208:56:352
182742099190,18cyclictest0-21swapper/212:15:352
182742099190,18cyclictest0-21swapper/209:18:322
182742099190,18cyclictest0-21swapper/208:52:432
182742099190,18cyclictest0-21swapper/208:12:342
182741599195,12cyclictest1915574-21ps11:15:201
1827415991917,1cyclictest3056-21PK-Backend09:16:301
1827415991916,2cyclictest251rcuc/109:41:081
1827415991916,2cyclictest0-21swapper/108:02:091
182741599190,1cyclictest0-21swapper/110:07:081
182741599190,1cyclictest0-21swapper/109:57:271
182741599190,18cyclictest0-21swapper/110:51:141
1827413991917,1cyclictest0-21swapper/010:12:460
1827413991917,1cyclictest0-21swapper/009:17:170
1827413991917,1cyclictest0-21swapper/007:34:560
182741399190,1cyclictest3056-21PK-Backend12:05:280
182741399190,1cyclictest0-21swapper/011:14:040
182741399190,1cyclictest0-21swapper/009:10:340
182741399190,1cyclictest0-21swapper/008:23:430
182741399190,1cyclictest0-21swapper/008:09:090
182741399190,18cyclictest0-21swapper/009:53:120
182741399190,18cyclictest0-21swapper/008:04:020
1827423991816,1cyclictest3056-21PK-Backend11:36:533
1827423991810,6cyclictest42-21ksoftirqd/307:40:273
182742399180,6cyclictest3056-21PK-Backend08:12:363
182742399180,5cyclictest3056-21PK-Backend10:45:163
1827420991816,1cyclictest0-21swapper/212:14:332
1827420991816,1cyclictest0-21swapper/209:37:102
1827420991816,1cyclictest0-21swapper/209:30:212
1827420991816,1cyclictest0-21swapper/209:27:312
1827420991816,1cyclictest0-21swapper/207:51:252
1827420991816,1cyclictest0-21swapper/207:15:572
182742099181,5cyclictest1938787-21sh12:25:012
1827420991812,4cyclictest883-21dbus-daemon11:18:032
1827420991811,6cyclictest1899260-21ls10:30:112
182742099180,1cyclictest0-21swapper/207:36:322
182742099180,17cyclictest0-21swapper/208:03:012
1827415991816,1cyclictest0-21swapper/110:37:161
1827415991816,1cyclictest0-21swapper/110:30:471
1827413991816,1cyclictest0-21swapper/007:45:510
1827413991814,3cyclictest1941350-21latency12:30:170
1827413991811,5cyclictest1928640-21awk11:55:110
1827413991810,6cyclictest1938081-21awk12:20:180
182741399180,16cyclictest1850468-21idleruntime-cro08:15:000
182742399171,7cyclictest1890957-21kworker/3:011:27:433
1827420991712,4cyclictest34-21ksoftirqd/212:39:592
1827420991712,4cyclictest34-21ksoftirqd/212:39:592
1827420991710,5cyclictest1917969-21cron11:25:022
182741399179,6cyclictest1897187-21apt10:25:120
182741399177,8cyclictest1936-21dbus-daemon12:28:490
1827413991715,1cyclictest0-21swapper/009:26:010
1827413991711,5cyclictest1833851-21ls07:25:170
182741399170,15cyclictest1937188-21sed12:20:010
1827420991614,1cyclictest0-21swapper/210:43:002
1827420991612,3cyclictest1861361-21/usr/sbin/munin08:45:192
1827420991612,3cyclictest1861361-21/usr/sbin/munin08:45:182
1827420991610,5cyclictest34-21ksoftirqd/208:20:172
1827420991610,5cyclictest1885080-21sed09:50:222
182741599162,9cyclictest1940810-21apt12:30:121
182741399164,3cyclictest141rcu_preempt12:40:000
182741399164,3cyclictest141rcu_preempt12:40:000
1827413991614,1cyclictest0-21swapper/008:47:220
1827413991614,1cyclictest0-21swapper/008:47:220
139099160,12rtkit-daemon1389-21rtkit-daemon08:27:490
182742099153,2cyclictest141rcu_preempt08:05:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional