You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-21 - 00:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed May 20, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
207194621180,5sleep33056-21PK-Backend08:45:013
139099820,66rtkit-daemon0-21swapper/107:05:141
203759326948,14sleep20-21swapper/207:08:332
203735826938,25sleep30-21swapper/307:05:333
203740526342,14sleep00-21swapper/007:06:100
21147352460,5sleep02114740-21cpuspeed_turbos10:45:120
20909602410,4sleep13056-21PK-Backend09:40:011
139099380,3rtkit-daemon0-21swapper/008:33:560
139099370,3rtkit-daemon0-21swapper/008:39:220
139099360,3rtkit-daemon0-21swapper/008:50:010
139099360,3rtkit-daemon0-21swapper/008:01:060
21544862340,2sleep00-21swapper/012:40:010
20476652290,4chrt3056-21PK-Backend07:35:123
203794199297,3cyclictest0-21swapper/210:05:002
20784562270,11sleep013-21ksoftirqd/009:01:100
2037929992716,8cyclictest436-20systemd-journal10:15:110
139099260,4rtkit-daemon1389-21rtkit-daemon09:55:153
203792999251,5cyclictest2156-21gnome-shell08:24:580
20600102240,3sleep33056-21PK-Backend08:10:133
203793599249,13cyclictest2084414-21cpuspeed_turbos09:20:111
139099240,17rtkit-daemon3056-21PK-Backend08:04:511
2037954992316,5cyclictest2081494-21ntp_kernel_pll_09:10:183
203795499230,21cyclictest2107012-21taskset10:21:083
2037954992220,1cyclictest3056-21PK-Backend10:01:063
2037954992220,1cyclictest0-21swapper/311:04:343
2037954992220,1cyclictest0-21swapper/307:49:263
2037941992220,1cyclictest0-21swapper/208:47:412
2037935992215,6cyclictest2068178-21apt-get08:33:111
2037935992214,6cyclictest2135333-21cron11:44:591
2037929992220,1cyclictest0-21swapper/009:36:310
2037929992220,1cyclictest0-21swapper/008:49:520
2037929992220,1cyclictest0-21swapper/008:42:570
2037929992220,1cyclictest0-21swapper/007:40:230
203792999221,2cyclictest2097866-21apt-get10:00:020
203792999220,20cyclictest3056-21PK-Backend07:35:230
203795499218,11cyclictest2119463-21cron10:59:593
2037954992119,1cyclictest3056-21PK-Backend09:00:433
2037954992119,1cyclictest3056-21PK-Backend08:56:043
2037954992119,1cyclictest0-21swapper/312:23:013
2037954992119,1cyclictest0-21swapper/312:13:163
2037954992119,1cyclictest0-21swapper/311:30:093
2037954992119,1cyclictest0-21swapper/311:26:173
2037954992119,1cyclictest0-21swapper/311:15:573
2037954992119,1cyclictest0-21swapper/311:07:253
2037954992119,1cyclictest0-21swapper/310:38:053
2037954992119,1cyclictest0-21swapper/310:31:493
2037954992119,1cyclictest0-21swapper/309:35:323
2037954992119,1cyclictest0-21swapper/309:33:283
2037954992119,1cyclictest0-21swapper/309:26:323
2037954992119,1cyclictest0-21swapper/307:26:213
2037954992118,2cyclictest978-21snmpd11:53:413
2037954992118,2cyclictest3056-21PK-Backend10:53:263
203795499211,17cyclictest2057313-21python308:00:213
203795499210,2cyclictest2108704-21chrt10:25:333
203795499210,20cyclictest0-21swapper/308:22:103
203795499210,20cyclictest0-21swapper/307:44:393
203795499210,19cyclictest3056-21PK-Backend08:52:123
2037941992119,1cyclictest3056-21PK-Backend11:45:462
2037941992119,1cyclictest0-21swapper/212:35:542
2037941992119,1cyclictest0-21swapper/212:32:502
2037941992119,1cyclictest0-21swapper/212:29:472
2037941992119,1cyclictest0-21swapper/212:16:562
2037941992119,1cyclictest0-21swapper/211:35:242
2037941992119,1cyclictest0-21swapper/210:56:252
2037941992119,1cyclictest0-21swapper/210:25:292
2037941992119,1cyclictest0-21swapper/210:17:112
2037941992119,1cyclictest0-21swapper/210:12:302
2037941992119,1cyclictest0-21swapper/209:23:442
2037941992119,1cyclictest0-21swapper/209:00:312
2037941992119,1cyclictest0-21swapper/208:51:472
2037941992119,1cyclictest0-21swapper/208:01:102
2037941992119,1cyclictest0-21swapper/207:48:532
2037941992117,3cyclictest34-21ksoftirqd/211:20:012
203794199210,2cyclictest2105736-21gpgv10:20:022
203794199210,20cyclictest0-21swapper/211:24:512
203794199210,20cyclictest0-21swapper/210:36:102
203794199210,20cyclictest0-21swapper/209:59:342
203794199210,20cyclictest0-21swapper/209:31:412
203794199210,20cyclictest0-21swapper/208:44:372
203794199210,19cyclictest0-21swapper/209:47:032
203794199210,19cyclictest0-21swapper/209:16:002
2037935992118,2cyclictest3056-21PK-Backend12:28:501
203793599210,19cyclictest3056-21PK-Backend11:11:161
203793599210,19cyclictest3056-21PK-Backend10:33:051
203793599210,19cyclictest3056-21PK-Backend10:25:341
203793599210,19cyclictest3056-21PK-Backend07:41:021
2037929992119,1cyclictest0-21swapper/012:15:480
2037929992119,1cyclictest0-21swapper/011:57:160
2037929992119,1cyclictest0-21swapper/011:09:590
2037929992119,1cyclictest0-21swapper/010:10:580
2037929992119,1cyclictest0-21swapper/009:53:470
2037929992119,1cyclictest0-21swapper/009:46:450
2037929992119,1cyclictest0-21swapper/009:40:360
2037929992119,1cyclictest0-21swapper/009:26:290
2037929992119,1cyclictest0-21swapper/009:22:370
2037929992119,1cyclictest0-21swapper/009:15:560
2037929992119,1cyclictest0-21swapper/009:14:540
2037929992119,1cyclictest0-21swapper/008:19:110
2037929992119,1cyclictest0-21swapper/007:51:120
2037929992119,1cyclictest0-21swapper/007:51:120
2037929992119,1cyclictest0-21swapper/007:13:060
2037929992118,2cyclictest3056-21PK-Backend11:46:430
203792999210,20cyclictest0-21swapper/012:10:560
139099210,14rtkit-daemon0-21swapper/107:31:291
2037954992018,1cyclictest3056-21PK-Backend10:41:433
2037954992018,1cyclictest3056-21PK-Backend07:33:383
2037954992018,1cyclictest3056-21PK-Backend07:12:483
2037954992018,1cyclictest0-21swapper/312:05:323
2037954992018,1cyclictest0-21swapper/312:03:283
2037954992018,1cyclictest0-21swapper/309:53:033
2037954992018,1cyclictest0-21swapper/309:48:373
2037954992018,1cyclictest0-21swapper/309:40:393
2037954992018,1cyclictest0-21swapper/308:18:403
2037954992016,3cyclictest2135607-21apt11:45:103
2037954992011,7cyclictest2150545-21awk12:25:223
203795499200,1cyclictest3056-21PK-Backend10:46:093
203795499200,19cyclictest3056-21PK-Backend10:16:283
203795499200,19cyclictest3056-21PK-Backend08:38:343
203795499200,19cyclictest0-21swapper/311:36:393
203795499200,19cyclictest0-21swapper/311:14:553
203795499200,19cyclictest0-21swapper/307:50:413
203795499200,19cyclictest0-21swapper/307:50:413
203795499200,18cyclictest3056-21PK-Backend10:07:233
2037941992018,1cyclictest0-21swapper/211:44:442
2037941992018,1cyclictest0-21swapper/210:41:372
2037941992018,1cyclictest0-21swapper/209:50:412
2037941992018,1cyclictest0-21swapper/209:35:472
2037941992018,1cyclictest0-21swapper/209:25:472
2037941992018,1cyclictest0-21swapper/208:34:222
2037941992018,1cyclictest0-21swapper/208:21:052
2037941992018,1cyclictest0-21swapper/208:16:182
2037941992018,1cyclictest0-21swapper/207:51:242
2037941992018,1cyclictest0-21swapper/207:51:232
2037941992018,1cyclictest0-21swapper/207:22:162
2037941992012,6cyclictest2080764-21awk09:10:092
203794199200,1cyclictest0-21swapper/210:48:552
203794199200,19cyclictest3056-21PK-Backend10:53:342
203794199200,19cyclictest0-21swapper/212:14:532
203794199200,19cyclictest0-21swapper/212:01:132
203794199200,19cyclictest0-21swapper/211:51:272
203794199200,19cyclictest0-21swapper/211:25:522
203794199200,19cyclictest0-21swapper/211:12:262
203794199200,19cyclictest0-21swapper/211:01:042
203794199200,19cyclictest0-21swapper/208:56:402
203794199200,19cyclictest0-21swapper/208:05:432
203794199200,19cyclictest0-21swapper/207:37:522
203794199200,19cyclictest0-21swapper/207:34:072
2037935992018,1cyclictest3056-21PK-Backend12:30:531
2037935992018,1cyclictest3056-21PK-Backend11:02:591
2037935992018,1cyclictest3056-21PK-Backend10:55:421
2037935992018,1cyclictest3056-21PK-Backend09:55:061
2037935992018,1cyclictest3056-21PK-Backend08:17:031
2037935992018,1cyclictest0-21swapper/111:23:271
2037935992018,1cyclictest0-21swapper/110:16:071
2037935992018,1cyclictest0-21swapper/110:01:211
2037935992018,1cyclictest0-21swapper/107:11:551
2037935992017,2cyclictest3056-21PK-Backend07:52:511
2037935992017,2cyclictest3056-21PK-Backend07:52:501
203793599200,1cyclictest3056-21PK-Backend11:47:331
203793599200,1cyclictest3056-21PK-Backend11:06:371
203793599200,19cyclictest3056-21PK-Backend10:20:471
203793599200,19cyclictest3056-21PK-Backend09:41:061
203793599200,19cyclictest3056-21PK-Backend09:10:571
203793599200,19cyclictest3056-21PK-Backend08:22:521
203793599200,19cyclictest3056-21PK-Backend07:38:311
203793599200,19cyclictest3056-21PK-Backend07:26:281
203793599200,19cyclictest0-21swapper/110:52:511
203793599200,19cyclictest0-21swapper/110:47:231
203793599200,19cyclictest0-21swapper/109:03:151
203793599200,19cyclictest0-21swapper/108:50:511
203793599200,19cyclictest0-21swapper/108:11:151
2037929992018,1cyclictest3056-21PK-Backend07:25:370
2037929992018,1cyclictest0-21swapper/012:23:320
2037929992018,1cyclictest0-21swapper/011:11:000
2037929992018,1cyclictest0-21swapper/010:36:130
2037929992018,1cyclictest0-21swapper/010:05:300
2037929992018,1cyclictest0-21swapper/008:28:430
2037929992018,1cyclictest0-21swapper/007:59:300
2037929992018,1cyclictest0-21swapper/007:59:300
2037929992018,1cyclictest0-21swapper/007:46:260
2037929992018,1cyclictest0-21swapper/007:15:100
2037929992017,2cyclictest2079902-21python309:05:210
2037929992017,2cyclictest0-21swapper/007:20:170
2037929992013,5cyclictest2097858-21cron09:59:590
203792999200,19cyclictest0-21swapper/012:32:320
203792999200,19cyclictest0-21swapper/012:06:030
203792999200,19cyclictest0-21swapper/012:01:090
203792999200,19cyclictest0-21swapper/011:52:230
203792999200,19cyclictest0-21swapper/011:24:590
203792999200,19cyclictest0-21swapper/009:32:250
2037954991917,1cyclictest3056-21PK-Backend10:12:503
2037954991917,1cyclictest0-21swapper/312:36:073
2037954991917,1cyclictest0-21swapper/312:33:023
2037954991917,1cyclictest0-21swapper/311:58:343
2037954991917,1cyclictest0-21swapper/308:05:013
2037954991914,3cyclictest978-21snmpd08:26:053
2037954991914,3cyclictest2067923-21users08:30:223
2037954991912,5cyclictest2082724-21cstates09:15:123
2037954991911,6cyclictest42-21ksoftirqd/309:08:093
203795499190,9cyclictest2063198-21kworker/3:211:20:143
203795499190,7cyclictest2056192-21munin-run08:00:003
203795499190,7cyclictest2056192-21munin-run07:59:593
203795499190,1cyclictest3056-21PK-Backend09:21:403
203795499190,1cyclictest3056-21PK-Backend07:18:433
2037941991917,1cyclictest0-21swapper/211:57:212
2037941991917,1cyclictest0-21swapper/209:40:542
2037941991917,1cyclictest0-21swapper/208:39:432
2037941991917,1cyclictest0-21swapper/207:58:412
2037941991917,1cyclictest0-21swapper/207:58:412
2037941991915,3cyclictest978-21snmpd07:15:372
2037941991912,5cyclictest2100232-21dpkg10:05:002
2037941991912,5cyclictest2066090-21sed08:25:232
2037941991911,6cyclictest2050714-21cron07:45:012
203794199190,1cyclictest0-21swapper/211:08:342
203794199190,1cyclictest0-21swapper/207:13:532
203794199190,18cyclictest0-21swapper/212:21:492
203794199190,18cyclictest0-21swapper/212:06:072
203794199190,18cyclictest0-21swapper/209:07:142
203794199190,18cyclictest0-21swapper/208:13:482
2037935991917,1cyclictest3056-21PK-Backend12:37:481
2037935991917,1cyclictest3056-21PK-Backend12:21:061
2037935991917,1cyclictest3056-21PK-Backend12:00:591
2037935991917,1cyclictest3056-21PK-Backend09:33:091
2037935991917,1cyclictest3056-21PK-Backend08:06:431
2037935991917,1cyclictest3056-21PK-Backend07:57:381
2037935991917,1cyclictest3056-21PK-Backend07:57:381
2037935991917,1cyclictest0-21swapper/108:40:021
2037935991916,2cyclictest3056-21PK-Backend12:07:401
2037935991916,2cyclictest3056-21PK-Backend09:46:131
2037935991913,5cyclictest2083929-21http09:19:221
2037935991913,4cyclictest2173-21JS10:27:481
2037935991912,5cyclictest2100964-21cat10:05:151
203793599190,1cyclictest3056-21PK-Backend12:10:311
203793599190,1cyclictest3056-21PK-Backend11:59:571
203793599190,1cyclictest3056-21PK-Backend11:50:241
203793599190,1cyclictest3056-21PK-Backend11:26:181
203793599190,1cyclictest3056-21PK-Backend09:53:161
203793599190,18cyclictest3056-21PK-Backend09:07:061
203793599190,18cyclictest0-21swapper/110:43:451
203793599190,18cyclictest0-21swapper/107:16:481
2037929991917,1cyclictest0-21swapper/012:25:350
2037929991917,1cyclictest0-21swapper/011:43:520
2037929991917,1cyclictest0-21swapper/010:27:190
2037929991912,5cyclictest2125385-21cat11:15:140
203792999190,1cyclictest0-21swapper/007:31:390
203792999190,18cyclictest0-21swapper/008:12:080
2037954991816,1cyclictest3056-21PK-Backend07:24:053
2037954991812,4cyclictest2134545-21strings11:40:183
2037954991811,5cyclictest436-20systemd-journal12:15:103
2037941991816,1cyclictest0-21swapper/211:32:342
2037941991816,1cyclictest0-21swapper/207:26:492
2037935991816,1cyclictest0-21swapper/108:58:351
2037935991816,1cyclictest0-21swapper/107:24:121
2037935991815,2cyclictest3056-21PK-Backend08:25:221
203793599180,1cyclictest3056-21PK-Backend07:48:051
2037929991816,1cyclictest0-21swapper/011:39:120
2037929991816,1cyclictest0-21swapper/011:27:010
2037929991816,1cyclictest0-21swapper/010:20:160
2037929991816,1cyclictest0-21swapper/008:58:370
2037929991814,3cyclictest0-21swapper/010:55:110
2037929991812,5cyclictest13-21ksoftirqd/010:40:120
2037929991811,5cyclictest2116258-21find10:50:100
2037954991715,1cyclictest3056-21PK-Backend08:48:213
2037941991715,1cyclictest0-21swapper/210:34:202
2037935991715,1cyclictest3056-21PK-Backend11:33:481
2037935991715,1cyclictest0-21swapper/110:11:271
2037935991714,2cyclictest3056-21PK-Backend12:15:241
2037935991712,3cyclictest2087004-21timerwakeupswit09:25:211
2037929991715,1cyclictest0-21swapper/011:03:300
2037929991715,1cyclictest0-21swapper/010:31:450
2037929991712,3cyclictest2131313-21switchtime11:30:220
2037935991614,1cyclictest3056-21PK-Backend10:36:431
2037935991614,1cyclictest0-21swapper/111:18:461
2037929991614,1cyclictest0-21swapper/008:09:510
203793599159,4cyclictest3056-21PK-Backend11:36:451
2037935991513,1cyclictest0-21swapper/108:45:561
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional