You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 15:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Jun 21, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
336106922990,29sleep013-21ksoftirqd/007:05:140
336141522720,8sleep20-21swapper/207:05:192
346657121270,4sleep13056-21PK-Backend12:05:011
343221921030,3sleep00-21swapper/010:25:150
34438582980,2sleep13056-21PK-Backend11:00:021
336085428241,32sleep10-21swapper/107:05:111
336206826343,14sleep30-21swapper/307:06:413
34779982560,4sleep20-21swapper/212:35:182
139099490,4rtkit-daemon1389-21rtkit-daemon11:11:512
34410842480,3chrt3056-21PK-Backend10:50:191
34249202470,2sleep10-21swapper/110:05:151
139099440,4rtkit-daemon1389-21rtkit-daemon08:17:092
34618902400,4sleep03461891-21kernelversion11:50:160
34233292300,13sleep126-21ksoftirqd/110:00:141
336246199270,25cyclictest1389-21rtkit-daemon07:10:130
139099260,4rtkit-daemon1389-21rtkit-daemon11:07:303
139099260,4rtkit-daemon1389-21rtkit-daemon10:35:303
139099260,4rtkit-daemon1389-21rtkit-daemon10:20:482
336247199258,15cyclictest886-21NetworkManager07:59:383
3362471992517,6cyclictest3454192-21cron11:30:003
336246999250,24cyclictest1389-21rtkit-daemon11:49:092
3362464992518,5cyclictest3399872-21cron08:55:001
3362471992420,3cyclictest42-21ksoftirqd/310:20:223
3362471992420,2cyclictest3377078-21ls07:50:003
3362471992416,6cyclictest883-21dbus-daemon10:15:023
3362461992416,6cyclictest3401866-21cron09:00:000
34298702230,4chrt3056-21PK-Backend10:20:100
33851212230,3sleep20-21swapper/208:10:172
336247199239,8cyclictest42-21ksoftirqd/311:15:183
336247199237,3cyclictest3429619-21latency_hist10:20:003
336247199231,9cyclictest3397549-21python308:45:193
3362471992312,7cyclictest42-21ksoftirqd/311:55:013
3362469992315,6cyclictest3371547-21cron07:35:002
336246999230,21cyclictest3403881-21apt09:05:122
336247199225,14cyclictest3412297-21idleruntime-cro09:30:003
336247199225,14cyclictest3412297-21idleruntime-cro09:30:003
3362471992219,2cyclictest3056-21PK-Backend12:16:143
3362471992219,2cyclictest3056-21PK-Backend12:14:523
3362471992219,2cyclictest3056-21PK-Backend09:14:173
3362471992219,2cyclictest3056-21PK-Backend09:03:093
3362471992213,7cyclictest436-20systemd-journal10:30:003
3362471992212,3cyclictest42-21ksoftirqd/311:00:013
336247199220,20cyclictest3056-21PK-Backend11:20:103
336247199220,20cyclictest3056-21PK-Backend09:55:273
336247199220,20cyclictest3056-21PK-Backend09:08:093
336246999220,3cyclictest3415239-21grep09:35:202
336246499220,3cyclictest3056-21PK-Backend12:00:001
3362461992213,7cyclictest3395323-21df_inode08:40:130
34624012210,4sleep13056-21PK-Backend11:50:211
3362471992119,1cyclictest3056-21PK-Backend10:09:203
3362471992119,1cyclictest3056-21PK-Backend09:33:173
3362471992119,1cyclictest3056-21PK-Backend09:17:223
3362471992119,1cyclictest3056-21PK-Backend08:58:093
3362471992119,1cyclictest3056-21PK-Backend07:21:433
3362471992119,1cyclictest0-21swapper/311:48:433
3362471992119,1cyclictest0-21swapper/310:53:483
3362471992119,1cyclictest0-21swapper/310:31:033
3362471992119,1cyclictest0-21swapper/308:05:503
3362471992119,1cyclictest0-21swapper/307:51:103
3362471992112,6cyclictest42-21ksoftirqd/308:15:213
3362471992112,3cyclictest42-21ksoftirqd/312:05:013
336247199211,17cyclictest3415148-21python309:35:193
3362471992111,3cyclictest42-21ksoftirqd/309:50:003
336247199210,20cyclictest3056-21PK-Backend08:26:463
336247199210,20cyclictest0-21swapper/309:53:313
336246999210,20cyclictest0-21swapper/209:48:522
3362464992118,2cyclictest0-21swapper/107:30:211
3362464992115,4cyclictest3366361-21latency_hist07:19:591
3362464992114,6cyclictest3375826-21expr07:45:111
336246499210,20cyclictest3056-21PK-Backend10:46:371
336246499210,19cyclictest3056-21PK-Backend11:05:301
336246499210,19cyclictest3056-21PK-Backend09:50:541
3362461992114,6cyclictest3404746-21swap09:05:220
3362461992113,6cyclictest3434061-21awk10:30:180
336246199210,3cyclictest3470960-21cat12:15:160
336246199210,20cyclictest0-21swapper/009:30:160
336246199210,20cyclictest0-21swapper/007:21:330
34483712200,4sleep13056-21PK-Backend11:10:201
3362471992018,1cyclictest0-21swapper/311:43:503
3362471992018,1cyclictest0-21swapper/311:34:043
3362471992018,1cyclictest0-21swapper/308:22:063
3362471992017,2cyclictest3056-21PK-Backend09:20:273
3362471992015,3cyclictest3448387-21python311:10:203
336247199201,16cyclictest3443549-21awk11:00:003
3362471992011,3cyclictest42-21ksoftirqd/311:35:193
3362471992010,7cyclictest42-21ksoftirqd/312:05:133
336247199200,1cyclictest3056-21PK-Backend07:15:343
336247199200,19cyclictest3056-21PK-Backend12:38:593
336247199200,19cyclictest3056-21PK-Backend12:26:563
336247199200,19cyclictest3056-21PK-Backend12:20:543
336247199200,19cyclictest3056-21PK-Backend08:53:093
336247199200,19cyclictest0-21swapper/309:43:173
336247199200,19cyclictest0-21swapper/308:43:563
336247199200,19cyclictest0-21swapper/308:14:083
336247199200,19cyclictest0-21swapper/307:39:013
336246999205,13cyclictest3391211-21cron08:29:592
3362469992018,1cyclictest0-21swapper/211:09:092
3362469992018,1cyclictest0-21swapper/208:31:262
3362469992018,1cyclictest0-21swapper/208:31:252
3362469992018,1cyclictest0-21swapper/207:46:102
3362469992017,2cyclictest0-21swapper/211:00:232
3362469992017,2cyclictest0-21swapper/207:21:102
3362469992014,5cyclictest3379375-21expr07:55:112
3362469992012,6cyclictest3406470-21python309:10:192
3362469992012,6cyclictest3388417-21fw_packets08:20:152
336246999200,1cyclictest0-21swapper/210:25:222
336246999200,19cyclictest0-21swapper/212:27:502
336246999200,19cyclictest0-21swapper/211:37:072
336246999200,19cyclictest0-21swapper/210:15:352
336246999200,19cyclictest0-21swapper/209:41:492
336246999200,19cyclictest0-21swapper/207:37:382
336246499204,14cyclictest3363184-21cat07:10:161
3362464992018,1cyclictest3056-21PK-Backend08:05:091
3362464992018,1cyclictest3056-21PK-Backend07:57:111
3362464992014,5cyclictest3391219-21grep08:29:591
3362464992013,6cyclictest3478757-21/usr/sbin/munin12:40:001
3362464992012,6cyclictest3409278-21grep09:20:111
336246499200,2cyclictest0-21swapper/107:45:011
336246499200,1cyclictest3056-21PK-Backend11:48:021
336246499200,1cyclictest3056-21PK-Backend08:17:541
336246499200,1cyclictest3056-21PK-Backend08:10:091
336246499200,19cyclictest0-21swapper/112:19:591
336246499200,19cyclictest0-21swapper/111:19:091
336246499200,19cyclictest0-21swapper/109:55:201
336246499200,19cyclictest0-21swapper/109:28:231
336246499200,19cyclictest0-21swapper/109:28:231
336246499200,19cyclictest0-21swapper/109:19:451
336246499200,19cyclictest0-21swapper/109:12:291
336246499200,19cyclictest0-21swapper/108:33:481
336246499200,19cyclictest0-21swapper/108:33:481
336246499200,18cyclictest3056-21PK-Backend10:11:561
3362461992018,1cyclictest0-21swapper/011:37:240
3362461992018,1cyclictest0-21swapper/008:53:330
3362461992018,1cyclictest0-21swapper/007:39:380
336246199201,5cyclictest13-21ksoftirqd/007:58:440
3362461992014,5cyclictest13-21ksoftirqd/009:55:100
3362461992013,5cyclictest883-21dbus-daemon11:10:010
3362461992012,6cyclictest3478359-21users12:35:210
3362461992011,7cyclictest2237-21gdbus09:14:400
336246199200,1cyclictest0-21swapper/010:19:510
336246199200,1cyclictest0-21swapper/008:12:020
336246199200,19cyclictest0-21swapper/012:12:330
336246199200,19cyclictest0-21swapper/011:33:530
336246199200,19cyclictest0-21swapper/010:57:570
336246199200,19cyclictest0-21swapper/009:49:090
336246199200,19cyclictest0-21swapper/009:02:590
336246199200,19cyclictest0-21swapper/008:29:070
336246199200,19cyclictest0-21swapper/007:43:090
336246199200,18cyclictest3427869-21cron10:15:000
3362471991917,1cyclictest0-21swapper/308:01:253
3362471991916,2cyclictest3056-21PK-Backend08:34:093
3362471991916,2cyclictest3056-21PK-Backend08:34:093
3362471991912,5cyclictest3439257-21ntpq10:45:193
3362471991912,2cyclictest42-21ksoftirqd/310:40:233
336247199190,1cyclictest3056-21PK-Backend07:43:533
336247199190,1cyclictest3056-21PK-Backend07:34:063
336247199190,1cyclictest3056-21PK-Backend07:10:153
3362469991917,1cyclictest0-21swapper/211:40:252
3362469991915,3cyclictest28650irq/123-ahci[0012:15:002
3362469991914,4cyclictest978-21snmpd07:51:072
3362469991913,5cyclictest3455131-21grep11:30:162
3362469991913,5cyclictest3426620-21seq10:10:142
3362469991913,5cyclictest3420357-21swap09:50:232
3362469991912,5cyclictest3402208-21cat09:00:142
336246999190,6cyclictest3464611-21munin-run12:00:002
336246999190,1cyclictest0-21swapper/211:21:392
336246999190,1cyclictest0-21swapper/209:31:552
336246999190,18cyclictest0-21swapper/210:41:142
336246999190,18cyclictest0-21swapper/207:29:172
336246999190,18cyclictest0-21swapper/207:29:172
336246999190,18cyclictest0-21swapper/207:15:282
336246499194,2cyclictest141rcu_preempt09:45:221
3362464991917,1cyclictest3056-21PK-Backend11:25:241
3362464991917,1cyclictest3056-21PK-Backend11:01:241
3362464991917,1cyclictest3056-21PK-Backend09:38:581
3362464991917,1cyclictest3056-21PK-Backend07:54:151
3362464991917,1cyclictest0-21swapper/109:30:531
3362464991917,1cyclictest0-21swapper/108:38:481
3362464991917,1cyclictest0-21swapper/107:27:381
3362464991917,1cyclictest0-21swapper/107:27:371
3362464991912,5cyclictest436-20systemd-journal09:04:551
336246499190,1cyclictest3056-21PK-Backend12:06:341
336246499190,1cyclictest3056-21PK-Backend10:18:181
336246499190,1cyclictest3056-21PK-Backend09:44:321
336246499190,1cyclictest3056-21PK-Backend08:21:111
336246499190,1cyclictest0-21swapper/108:48:351
336246499190,18cyclictest3056-21PK-Backend10:43:201
336246499190,18cyclictest3056-21PK-Backend10:33:261
336246499190,18cyclictest0-21swapper/109:09:581
336246499190,18cyclictest0-21swapper/108:56:401
3362461991917,1cyclictest0-21swapper/011:57:320
3362461991917,1cyclictest0-21swapper/011:00:070
3362461991917,1cyclictest0-21swapper/009:53:010
3362461991917,1cyclictest0-21swapper/009:21:390
3362461991917,1cyclictest0-21swapper/008:47:590
3362461991917,1cyclictest0-21swapper/008:03:100
3362461991914,3cyclictest3475923-21sh12:30:130
3362461991914,3cyclictest3394681-21dump-pmu-power08:40:000
3362461991913,5cyclictest3425138-21if_err_enp2s010:05:190
3362461991912,5cyclictest3441784-21grep10:55:010
3362461991912,5cyclictest3392197-21ntpq08:30:190
3362461991912,5cyclictest3392197-21ntpq08:30:190
3362461991911,6cyclictest3407715-21df09:15:110
336246199190,1cyclictest0-21swapper/011:14:330
336246199190,1cyclictest0-21swapper/009:42:580
336246199190,1cyclictest0-21swapper/007:32:010
336246199190,1cyclictest0-21swapper/007:15:240
336246199190,18cyclictest0-21swapper/012:20:440
336246199190,18cyclictest0-21swapper/012:06:310
336246199190,18cyclictest0-21swapper/010:36:350
336246199190,18cyclictest0-21swapper/008:24:260
336246199190,18cyclictest0-21swapper/007:29:170
336246199190,18cyclictest0-21swapper/007:29:170
3362471991815,2cyclictest3056-21PK-Backend07:26:093
3362471991815,2cyclictest3056-21PK-Backend07:26:093
336247199180,5cyclictest3464077-21awk11:55:203
336246999188,8cyclictest0-21swapper/207:10:092
336246999185,3cyclictest141rcu_preempt07:45:022
336246999184,3cyclictest141rcu_preempt09:30:012
336246999184,3cyclictest141rcu_preempt09:30:002
3362469991814,3cyclictest34-21ksoftirqd/211:55:012
3362469991813,3cyclictest3441311-21timerwakeupswit10:50:202
3362469991813,3cyclictest1095-21runrttasks10:45:122
3362469991812,5cyclictest3443032-21cat10:55:222
3362469991811,5cyclictest3434092-21ntpq10:30:192
336246499188,7cyclictest3433081-21sh10:30:011
336246499184,2cyclictest141rcu_preempt11:25:011
3362464991816,1cyclictest3056-21PK-Backend11:34:571
3362464991816,1cyclictest0-21swapper/112:25:271
3362464991816,1cyclictest0-21swapper/108:41:541
3362464991816,1cyclictest0-21swapper/107:39:541
3362464991816,1cyclictest0-21swapper/107:24:191
3362464991813,3cyclictest3475680-21tr12:30:121
3362464991812,4cyclictest3430009-21cpuspeed_turbos10:20:121
3362464991811,5cyclictest3472434-21expr12:20:121
336246499180,1cyclictest0-21swapper/112:12:361
3362461991816,1cyclictest0-21swapper/012:27:330
3362461991816,1cyclictest0-21swapper/011:43:400
3362461991816,1cyclictest0-21swapper/011:16:430
3362461991816,1cyclictest0-21swapper/010:44:320
3362461991816,1cyclictest0-21swapper/010:01:190
3362461991816,1cyclictest0-21swapper/009:37:470
3362461991816,1cyclictest0-21swapper/009:27:460
3362461991816,1cyclictest0-21swapper/009:27:460
3362461991816,1cyclictest0-21swapper/008:15:200
3362461991816,1cyclictest0-21swapper/008:05:060
3362461991812,5cyclictest3461237-21cut11:50:010
3362461991810,6cyclictest3451657-21grep11:20:170
34256212170,2sleep20-21swapper/210:05:242
336246999174,6cyclictest141rcu_preempt10:00:222
336246999174,3cyclictest141rcu_preempt10:00:002
336246999174,2cyclictest141rcu_preempt12:15:142
336246999174,2cyclictest141rcu_preempt11:15:132
336246999173,7cyclictest141rcu_preempt12:20:182
336246999173,7cyclictest141rcu_preempt09:00:002
336246999173,3cyclictest141rcu_preempt09:25:002
3362469991712,4cyclictest978-21snmpd12:07:262
3362469991711,5cyclictest3381749-21ls08:00:222
3362469991711,4cyclictest436-20systemd-journal08:41:222
3362469991711,4cyclictest3399009-21grep08:50:172
3362469991710,5cyclictest3466510-21cmp12:05:012
3362469991710,5cyclictest3408832-21perf09:19:592
336246999170,1cyclictest0-21swapper/212:33:252
336246999170,16cyclictest0-21swapper/211:27:272
3362461991715,1cyclictest0-21swapper/010:49:110
3362461991715,1cyclictest0-21swapper/007:49:380
3362471991614,1cyclictest3056-21PK-Backend12:34:193
336246999169,5cyclictest3397412-21ntpq08:45:172
336246999161,3cyclictest3384063-21munin-run08:10:022
3362464991614,1cyclictest3056-21PK-Backend11:41:121
3362464991614,1cyclictest3056-21PK-Backend08:02:391
3362461991614,1cyclictest0-21swapper/012:01:040
3362461991614,1cyclictest0-21swapper/007:54:450
3362461991611,4cyclictest3453019-21expr11:25:120
336246999158,5cyclictest3393715-21cut08:35:172
3362469991513,1cyclictest0-21swapper/210:39:352
3362464991513,1cyclictest3056-21PK-Backend10:38:061
336246499150,6cyclictest3056-21PK-Backend11:35:211
336247199149,4cyclictest3056-21PK-Backend10:03:213
3362471991410,3cyclictest3056-21PK-Backend08:39:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional