You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-18 - 19:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Aug 18, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
265308921350,4sleep02653087-21timerandwakeup10:40:250
257663929845,15sleep00-21swapper/007:05:230
257722427940,14sleep20-21swapper/207:08:252
257721826546,13sleep10-21swapper/107:08:201
257711726444,13sleep30-21swapper/307:07:033
25982942510,4sleep33056-21PK-Backend08:05:163
26292792480,8sleep126-21ksoftirqd/109:35:001
257750099442,28cyclictest2625873-21apt09:25:140
26221602430,3sleep10-21swapper/109:14:591
26163292410,3sleep00-21swapper/008:55:220
26020732400,3sleep00-21swapper/008:15:160
257750099322,18cyclictest2583364-21taskset07:24:550
257750099313,15cyclictest0-21swapper/008:32:580
2577500992813,13cyclictest2686608-21turbostat.cron12:20:000
26301932270,7sleep234-21ksoftirqd/209:35:222
257750099279,16cyclictest2631974-21sed09:40:210
257750099271,22cyclictest2687134-21apt12:20:120
2577500992712,13cyclictest2691489-21ls12:30:170
2577500992711,14cyclictest2645574-21df_inode10:20:140
2577511992614,9cyclictest883-21dbus-daemon10:42:053
257750099266,18cyclictest2679683-21ls12:00:010
257750099266,18cyclictest2590919-21dpkg07:45:110
257750099261,21cyclictest2577691-21apt07:10:120
2577500992611,13cyclictest2604056-21ntpq08:20:190
257750099258,15cyclictest2681910-21cat12:05:120
257750099258,15cyclictest2618425-21http09:02:040
257750099256,18cyclictest2689265-21which12:25:150
257750099256,17cyclictest2692865-21cat12:35:120
257750099253,6cyclictest2601416-21wc08:15:010
257750099252,22cyclictest2646916-21acpi10:25:120
2577500992511,13cyclictest2593989-21sshd07:53:410
2577500992510,14cyclictest2636003-21cron09:55:010
257750099250,3cyclictest2621048-21diskstats09:10:130
2577509992415,7cyclictest2361-21gdbus09:02:032
257750099249,14cyclictest2596772-21cut08:00:190
257750099249,13cyclictest2619920-21wc09:05:230
257750099247,15cyclictest2642931-21cron10:14:590
257750099247,15cyclictest0-21swapper/011:35:230
257750099246,16cyclictest2629284-21cut09:35:000
2577500992412,10cyclictest2633548-21ntp_states09:45:190
257750099241,10cyclictest2636283-21apt09:55:130
257750099240,22cyclictest2625612-21idleruntime-cro09:25:000
257750099240,22cyclictest2625612-21idleruntime-cro09:24:590
139099240,16rtkit-daemon3056-21PK-Backend10:51:541
26325562230,10chrt2632555-21basename09:45:012
2577509992321,1cyclictest3056-21PK-Backend07:50:172
2577509992321,1cyclictest0-21swapper/212:24:402
257750099238,13cyclictest2660032-21timerandwakeup11:00:220
257750099237,14cyclictest2584263-21missed_timers07:25:180
257750099236,15cyclictest2598746-21sed08:05:220
257750099234,4cyclictest2629448-21grep09:35:130
257750099231,19cyclictest2679938-21apt12:00:120
139099230,16rtkit-daemon3056-21PK-Backend11:37:301
2577511992220,1cyclictest0-21swapper/311:02:513
2577511992220,1cyclictest0-21swapper/310:58:323
2577511992220,1cyclictest0-21swapper/308:49:213
2577511992220,1cyclictest0-21swapper/308:27:173
2577511992219,2cyclictest3056-21PK-Backend10:22:303
257751199220,20cyclictest3056-21PK-Backend11:57:263
257751199220,20cyclictest3056-21PK-Backend10:25:133
257751199220,20cyclictest3056-21PK-Backend08:58:403
257751199220,20cyclictest0-21swapper/312:38:163
2577509992220,1cyclictest0-21swapper/209:09:182
2577509992217,3cyclictest2636002-21cron09:55:002
2577509992214,6cyclictest2658215-21python310:55:222
257750999220,21cyclictest0-21swapper/209:34:542
2577504992220,1cyclictest3056-21PK-Backend11:22:521
2577504992220,1cyclictest0-21swapper/111:59:081
2577504992220,1cyclictest0-21swapper/110:29:251
2577504992219,2cyclictest3056-21PK-Backend11:26:241
2577504992218,2cyclictest2582662-21ntp_states07:20:181
2577504992217,3cyclictest26-21ksoftirqd/108:16:591
257750099228,12cyclictest2662864-21expr11:10:130
257750099228,12cyclictest2613938-21cstates08:50:120
257750099226,14cyclictest0-21swapper/008:35:160
257750099225,15cyclictest26621242chrt11:07:350
257750099225,15cyclictest0-21swapper/010:55:010
257750099223,17cyclictest1095-21runrttasks10:08:280
257750099222,4cyclictest2643298-21dpkg10:15:010
257750099222,17cyclictest2666086-21wc11:20:120
2577500992215,5cyclictest2595048-21strings07:55:180
257750099221,19cyclictest2664463-21tr11:15:130
257750099221,18cyclictest2670403-21python311:30:220
257750099220,20cyclictest2589855-21cut07:40:180
257750099220,12cyclictest2164-21gdbus08:42:010
257750099220,10cyclictest2654758-21swap10:45:230
139099220,3rtkit-daemon0-21swapper/110:14:551
257751199216,14cyclictest0-21swapper/311:25:223
2577511992119,1cyclictest3056-21PK-Backend09:39:093
2577511992119,1cyclictest3056-21PK-Backend09:05:313
2577511992119,1cyclictest3056-21PK-Backend07:28:373
2577511992119,1cyclictest0-21swapper/312:09:503
2577511992119,1cyclictest0-21swapper/311:42:193
2577511992119,1cyclictest0-21swapper/310:31:283
2577511992119,1cyclictest0-21swapper/308:39:283
2577511992119,1cyclictest0-21swapper/308:20:553
2577511992119,1cyclictest0-21swapper/307:38:383
2577511992118,2cyclictest3056-21PK-Backend09:19:223
2577511992118,2cyclictest0-21swapper/310:02:493
257751199211,18cyclictest2656477-21python310:50:203
257751199210,2cyclictest0-21swapper/310:47:453
257751199210,20cyclictest3056-21PK-Backend12:34:303
257751199210,20cyclictest3056-21PK-Backend10:08:503
257751199210,20cyclictest3056-21PK-Backend07:46:563
2577509992119,1cyclictest0-21swapper/212:39:542
2577509992119,1cyclictest0-21swapper/212:26:152
2577509992119,1cyclictest0-21swapper/212:11:012
2577509992119,1cyclictest0-21swapper/211:55:262
2577509992119,1cyclictest0-21swapper/211:52:292
2577509992119,1cyclictest0-21swapper/211:35:462
2577509992119,1cyclictest0-21swapper/211:07:412
2577509992119,1cyclictest0-21swapper/210:42:332
2577509992119,1cyclictest0-21swapper/210:04:492
2577509992119,1cyclictest0-21swapper/209:27:232
2577509992119,1cyclictest0-21swapper/209:19:192
2577509992119,1cyclictest0-21swapper/209:11:292
2577509992119,1cyclictest0-21swapper/208:44:112
2577509992119,1cyclictest0-21swapper/207:47:062
2577509992119,1cyclictest0-21swapper/207:27:122
257750999210,20cyclictest0-21swapper/211:32:492
257750999210,20cyclictest0-21swapper/210:16:302
257750999210,20cyclictest0-21swapper/209:56:022
257750999210,20cyclictest0-21swapper/208:47:292
257750999210,20cyclictest0-21swapper/208:19:032
257750999210,20cyclictest0-21swapper/207:34:142
257750999210,20cyclictest0-21swapper/207:14:012
2577504992119,1cyclictest3056-21PK-Backend11:12:171
2577504992119,1cyclictest3056-21PK-Backend10:58:121
2577504992119,1cyclictest0-21swapper/112:18:151
2577504992119,1cyclictest0-21swapper/112:07:121
2577504992119,1cyclictest0-21swapper/109:27:061
2577504992119,1cyclictest0-21swapper/108:20:141
2577504992119,1cyclictest0-21swapper/108:11:351
2577504992119,1cyclictest0-21swapper/107:57:161
2577504992119,1cyclictest0-21swapper/107:26:211
2577504992114,5cyclictest2692294-21sshd12:34:411
257750499210,20cyclictest0-21swapper/109:46:471
257750499210,20cyclictest0-21swapper/107:46:341
257750499210,19cyclictest3056-21PK-Backend08:39:061
257750499210,19cyclictest3056-21PK-Backend08:25:151
257750099217,13cyclictest0-21swapper/010:50:210
257750099217,12cyclictest2676665-21cstates11:50:140
257750099215,14cyclictest2651021-21missed_timers10:35:190
257750099211,9cyclictest2668550-21ntp_states11:25:190
2577500992119,1cyclictest0-21swapper/012:10:100
257750099211,5cyclictest2580236-21expr07:15:120
257750099211,3cyclictest2156-21gnome-shell11:41:090
257750099211,3cyclictest0-21swapper/009:15:020
257750099210,3cyclictest25868702sleep007:31:580
139099210,14rtkit-daemon0-21swapper/109:16:561
139099210,14rtkit-daemon0-21swapper/107:12:531
2577511992018,1cyclictest3056-21PK-Backend12:25:393
2577511992018,1cyclictest3056-21PK-Backend12:24:513
2577511992018,1cyclictest3056-21PK-Backend12:02:343
2577511992018,1cyclictest3056-21PK-Backend08:18:263
2577511992018,1cyclictest3056-21PK-Backend07:32:083
2577511992018,1cyclictest0-21swapper/311:15:013
2577511992018,1cyclictest0-21swapper/311:12:523
2577511992018,1cyclictest0-21swapper/311:07:113
2577511992018,1cyclictest0-21swapper/310:38:313
2577511992017,2cyclictest3056-21PK-Backend10:17:023
2577511992017,2cyclictest3056-21PK-Backend08:10:353
257751199201,5cyclictest2675442-21runrttasks11:45:183
257751199200,19cyclictest3056-21PK-Backend09:56:483
257751199200,19cyclictest3056-21PK-Backend09:44:573
257751199200,19cyclictest3056-21PK-Backend09:00:033
257751199200,19cyclictest3056-21PK-Backend07:42:233
257751199200,19cyclictest0-21swapper/312:12:473
257751199200,19cyclictest0-21swapper/311:20:423
2577509992018,1cyclictest0-21swapper/212:01:212
2577509992018,1cyclictest0-21swapper/210:51:122
2577509992018,1cyclictest0-21swapper/210:35:302
2577509992018,1cyclictest0-21swapper/210:21:242
2577509992018,1cyclictest0-21swapper/208:52:082
2577509992018,1cyclictest0-21swapper/208:07:192
2577509992012,6cyclictest883-21dbus-daemon11:05:002
257750999200,19cyclictest3056-21PK-Backend07:20:232
257750999200,19cyclictest0-21swapper/211:42:162
257750999200,19cyclictest0-21swapper/211:26:202
257750999200,19cyclictest0-21swapper/211:13:222
257750999200,19cyclictest0-21swapper/208:59:312
257750999200,19cyclictest0-21swapper/207:44:422
257750999200,18cyclictest3056-21PK-Backend08:10:242
2577504992018,1cyclictest3056-21PK-Backend10:36:291
2577504992018,1cyclictest3056-21PK-Backend10:31:351
2577504992018,1cyclictest0-21swapper/112:22:471
2577504992018,1cyclictest0-21swapper/112:02:051
2577504992018,1cyclictest0-21swapper/110:16:201
2577504992018,1cyclictest0-21swapper/109:51:261
2577504992018,1cyclictest0-21swapper/109:40:591
2577504992018,1cyclictest0-21swapper/109:20:581
2577504992018,1cyclictest0-21swapper/109:20:581
2577504992018,1cyclictest0-21swapper/109:05:431
2577504992018,1cyclictest0-21swapper/108:59:291
2577504992018,1cyclictest0-21swapper/107:42:011
2577504992018,1cyclictest0-21swapper/107:15:591
2577504992017,2cyclictest3056-21PK-Backend11:00:211
257750499200,1cyclictest0-21swapper/109:58:011
257750499200,1cyclictest0-21swapper/107:33:581
257750499200,19cyclictest3056-21PK-Backend12:11:331
257750499200,19cyclictest3056-21PK-Backend08:32:451
257750499200,19cyclictest0-21swapper/109:03:331
257750099203,16cyclictest0-21swapper/011:45:360
257750099201,3cyclictest2156-21gnome-shell07:38:090
257750099201,3cyclictest0-21swapper/008:25:240
257750099200,5cyclictest945-21thermald10:00:260
257750099200,3cyclictest2649153-21irqstats10:30:170
139099200,14rtkit-daemon3056-21PK-Backend08:51:411
257751199193,14cyclictest0-21swapper/308:52:573
2577511991917,1cyclictest3056-21PK-Backend12:17:203
2577511991917,1cyclictest3056-21PK-Backend11:51:453
2577511991917,1cyclictest3056-21PK-Backend09:31:403
2577511991917,1cyclictest3056-21PK-Backend08:03:183
2577511991917,1cyclictest3056-21PK-Backend07:16:213
2577511991917,1cyclictest0-21swapper/311:31:313
2577511991917,1cyclictest0-21swapper/309:25:323
2577511991910,7cyclictest436-20systemd-journal08:45:013
257751199190,1cyclictest0-21swapper/311:37:133
257751199190,1cyclictest0-21swapper/308:31:433
2577509991917,1cyclictest3056-21PK-Backend08:04:152
2577509991917,1cyclictest0-21swapper/212:05:072
2577509991917,1cyclictest0-21swapper/210:46:522
2577509991917,1cyclictest0-21swapper/209:21:492
2577509991917,1cyclictest0-21swapper/209:21:492
2577509991917,1cyclictest0-21swapper/208:23:292
257750999190,1cyclictest0-21swapper/212:17:442
257750999190,1cyclictest0-21swapper/210:08:542
257750999190,1cyclictest0-21swapper/208:37:342
257750999190,1cyclictest0-21swapper/208:25:252
257750999190,1cyclictest0-21swapper/207:15:022
2577504991917,1cyclictest3056-21PK-Backend11:46:591
2577504991917,1cyclictest3056-21PK-Backend11:42:391
2577504991917,1cyclictest3056-21PK-Backend11:33:261
2577504991917,1cyclictest3056-21PK-Backend10:00:451
2577504991916,1cyclictest3056-21PK-Backend08:40:281
2577504991913,4cyclictest2156-21gnome-shell08:07:091
257750499190,1cyclictest3056-21PK-Backend11:15:481
257750499190,1cyclictest3056-21PK-Backend10:46:141
257750499190,1cyclictest3056-21PK-Backend10:41:211
257750499190,1cyclictest0-21swapper/111:05:141
257750499190,1cyclictest0-21swapper/110:24:311
257750499190,1cyclictest0-21swapper/109:35:101
257750499190,18cyclictest0-21swapper/110:07:211
257750099190,7cyclictest2612489-21grep08:45:170
139099190,13rtkit-daemon0-21swapper/112:35:281
2577511991816,1cyclictest3056-21PK-Backend10:12:563
2577511991816,1cyclictest3056-21PK-Backend09:53:303
2577511991813,3cyclictest2593508-21python307:50:213
2577509991816,1cyclictest0-21swapper/212:32:252
2577509991816,1cyclictest0-21swapper/210:33:202
2577509991816,1cyclictest0-21swapper/210:11:032
2577509991813,3cyclictest2664981-21needreboot11:15:192
2577509991812,5cyclictest2607216-21if_enp2s008:30:162
2577509991811,5cyclictest2666464-21forks11:20:152
2577504991816,1cyclictest3056-21PK-Backend07:36:541
2577504991816,1cyclictest0-21swapper/108:04:191
2577504991811,5cyclictest883-21dbus-daemon12:28:031
257751199174,11cyclictest2622135-21munin-run09:14:593
2577511991715,1cyclictest3056-21PK-Backend09:46:193
2577511991715,1cyclictest3056-21PK-Backend07:21:133
2577511991715,1cyclictest3056-21PK-Backend07:13:303
257750999179,6cyclictest436-20systemd-journal11:50:002
2577504991712,3cyclictest2593375-21ntp_offset07:50:191
257751199169,5cyclictest42-21ksoftirqd/309:20:223
257751199169,5cyclictest42-21ksoftirqd/309:20:223
2577509991614,1cyclictest0-21swapper/207:59:292
2577509991611,3cyclictest25881892sleep207:35:212
257750999160,4cyclictest0-21swapper/209:48:352
2577504991614,1cyclictest0-21swapper/111:51:171
257751199156,7cyclictest2595848-21cron08:00:003
257750999151,6cyclictest2648128-21sshd10:26:432
2577504991513,1cyclictest3056-21PK-Backend08:45:421
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional