You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 01:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Sep 02, 2026 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
267248622500,9sleep20-21swapper/207:10:002
270216721300,3sleep30-21swapper/308:30:173
267933321210,10sleep342-21ksoftirqd/307:25:213
139099870,75rtkit-daemon0-21swapper/007:06:390
267224827338,29sleep10-21swapper/107:07:391
267212626444,14sleep30-21swapper/307:06:163
27050572550,4sleep0161rcuc/008:40:000
27661282510,1sleep10-21swapper/111:30:201
26795052450,3sleep00-21swapper/007:25:220
27864382410,4sleep33056-21PK-Backend12:30:133
27013702380,4sleep33056-21PK-Backend08:30:013
27287562360,1chrt2728757-21/usr/sbin/munin09:45:153
26797952310,3sleep1251rcuc/107:28:271
27645362280,12sleep126-21ksoftirqd/111:25:221
267254399260,24cyclictest2760569-21sh11:15:210
27530912240,4chrt0-21swapper/010:55:140
27248202240,2sleep30-21swapper/309:35:013
2672556992319,2cyclictest0-21swapper/311:02:483
2672552992315,6cyclictest2706768-21apt-get08:45:002
2672552992314,7cyclictest2701067-21apt-get08:30:002
2672547992321,1cyclictest3056-21PK-Backend08:11:301
267254799230,22cyclictest0-21swapper/109:53:441
139099230,16rtkit-daemon3056-21PK-Backend07:50:163
139099230,14rtkit-daemon0-21swapper/211:52:342
2672556992220,1cyclictest3056-21PK-Backend11:52:233
2672556992220,1cyclictest0-21swapper/310:46:263
2672556992220,1cyclictest0-21swapper/310:36:393
267255699220,2cyclictest2758090-21apt11:10:123
267255699220,20cyclictest3056-21PK-Backend10:10:023
267255299227,3cyclictest27050602basename08:40:002
2672547992220,1cyclictest0-21swapper/111:16:311
2672547992220,1cyclictest0-21swapper/108:36:041
2672547992219,2cyclictest3056-21PK-Backend07:57:591
2672547992219,2cyclictest0-21swapper/111:20:441
2672547992215,4cyclictest2775437-21unin-run12:00:001
2672547992213,7cyclictest2782605-21cron12:20:001
2672547992213,6cyclictest2787809-21apt-get12:35:011
2672547992212,7cyclictest1075-21whoopsie10:02:211
267254799220,20cyclictest3056-21PK-Backend10:57:111
267254399220,3cyclictest27715652sleep011:45:220
139099220,14rtkit-daemon0-21swapper/209:30:472
2672556992119,1cyclictest3056-21PK-Backend09:50:293
2672556992119,1cyclictest3056-21PK-Backend09:23:453
2672556992119,1cyclictest3056-21PK-Backend09:18:453
2672556992119,1cyclictest3056-21PK-Backend08:59:253
2672556992119,1cyclictest0-21swapper/312:23:263
2672556992119,1cyclictest0-21swapper/308:17:143
2672556992119,1cyclictest0-21swapper/308:11:263
2672556992119,1cyclictest0-21swapper/307:56:123
2672556992119,1cyclictest0-21swapper/307:20:433
2672556992119,1cyclictest0-21swapper/307:15:293
2672556992118,2cyclictest3056-21PK-Backend12:39:003
2672556992118,2cyclictest3056-21PK-Backend11:48:113
2672556992118,2cyclictest3056-21PK-Backend11:28:233
267255699210,20cyclictest0-21swapper/308:07:563
267255699210,19cyclictest3056-21PK-Backend12:13:253
267255699210,19cyclictest3056-21PK-Backend11:36:353
267255699210,19cyclictest3056-21PK-Backend09:25:413
267255699210,19cyclictest3056-21PK-Backend07:32:263
267255299210,20cyclictest0-21swapper/211:21:252
267255299210,19cyclictest0-21swapper/211:38:152
267255299210,18cyclictest0-21swapper/210:47:252
2672547992119,1cyclictest3056-21PK-Backend09:49:591
2672547992119,1cyclictest3056-21PK-Backend08:03:051
2672547992119,1cyclictest0-21swapper/112:03:501
2672547992119,1cyclictest0-21swapper/109:55:211
2672547992119,1cyclictest0-21swapper/109:55:201
2672547992119,1cyclictest0-21swapper/109:29:311
2672547992119,1cyclictest0-21swapper/109:22:411
2672547992119,1cyclictest0-21swapper/109:10:241
2672547992119,1cyclictest0-21swapper/109:00:381
2672547992119,1cyclictest0-21swapper/108:41:251
2672547992119,1cyclictest0-21swapper/108:23:071
2672547992119,1cyclictest0-21swapper/107:32:571
2672547992110,8cyclictest883-21dbus-daemon09:35:011
267254799210,20cyclictest0-21swapper/111:36:321
267254799210,20cyclictest0-21swapper/108:56:251
267254799210,20cyclictest0-21swapper/107:40:411
267254799210,19cyclictest3056-21PK-Backend08:19:011
2672543992119,1cyclictest0-21swapper/012:04:150
2672543992119,1cyclictest0-21swapper/007:57:500
267254399210,20cyclictest0-21swapper/012:20:310
139099210,15rtkit-daemon3056-21PK-Backend11:34:223
139099210,15rtkit-daemon3056-21PK-Backend11:20:403
139099210,15rtkit-daemon3056-21PK-Backend10:09:113
139099210,15rtkit-daemon0-21swapper/210:09:042
139099210,14rtkit-daemon0-21swapper/208:58:092
27695872200,4sleep30-21swapper/311:40:203
27142072200,2sleep00-21swapper/009:05:110
2672556992018,1cyclictest3056-21PK-Backend12:16:503
2672556992018,1cyclictest3056-21PK-Backend09:57:113
2672556992018,1cyclictest3056-21PK-Backend09:57:113
2672556992018,1cyclictest3056-21PK-Backend09:35:423
2672556992018,1cyclictest3056-21PK-Backend09:02:303
2672556992018,1cyclictest0-21swapper/310:20:103
2672556992018,1cyclictest0-21swapper/308:04:373
2672556992018,1cyclictest0-21swapper/307:11:173
2672556992017,2cyclictest3056-21PK-Backend12:07:373
2672556992017,2cyclictest3056-21PK-Backend08:35:403
2672556992011,7cyclictest42-21ksoftirqd/310:30:123
267255699200,1cyclictest3056-21PK-Backend08:42:223
267255699200,19cyclictest3056-21PK-Backend09:40:023
267255699200,19cyclictest0-21swapper/312:00:003
267255699200,19cyclictest0-21swapper/308:50:063
2672552992018,1cyclictest3056-21PK-Backend08:45:182
2672552992018,1cyclictest0-21swapper/211:48:292
2672552992018,1cyclictest0-21swapper/211:41:402
2672552992018,1cyclictest0-21swapper/209:47:022
2672552992018,1cyclictest0-21swapper/209:13:022
2672552992018,1cyclictest0-21swapper/208:52:202
2672552992018,1cyclictest0-21swapper/207:55:152
2672552992018,1cyclictest0-21swapper/207:18:182
2672552992017,2cyclictest3056-21PK-Backend09:00:112
2672552992017,2cyclictest0-21swapper/212:37:162
2672552992017,2cyclictest0-21swapper/212:13:362
2672552992017,2cyclictest0-21swapper/209:09:242
2672552992014,4cyclictest2156-21gnome-shell11:11:092
2672552992013,5cyclictest34-21ksoftirqd/208:15:132
2672552992012,6cyclictest883-21dbus-daemon11:00:002
2672552992012,6cyclictest2749298-21cut10:45:012
267255299200,19cyclictest0-21swapper/211:09:352
267255299200,19cyclictest0-21swapper/211:01:242
267255299200,19cyclictest0-21swapper/209:21:062
267255299200,19cyclictest0-21swapper/207:31:022
267255299200,19cyclictest0-21swapper/207:20:072
267255299200,18cyclictest0-21swapper/209:51:362
2672547992018,1cyclictest3056-21PK-Backend10:54:011
2672547992018,1cyclictest3056-21PK-Backend10:21:151
2672547992018,1cyclictest0-21swapper/108:26:381
2672547992014,4cyclictest2708018-21ntp_kernel_pll_08:45:181
2672547992011,7cyclictest26-21ksoftirqd/111:45:091
267254799200,19cyclictest3056-21PK-Backend11:04:211
267254799200,19cyclictest0-21swapper/112:35:061
267254799200,19cyclictest0-21swapper/112:25:051
267254799200,19cyclictest0-21swapper/109:36:471
267254799200,19cyclictest0-21swapper/109:18:151
267254799200,19cyclictest0-21swapper/107:37:511
2672543992018,1cyclictest0-21swapper/012:14:290
2672543992018,1cyclictest0-21swapper/010:54:460
2672543992018,1cyclictest0-21swapper/010:18:430
2672543992018,1cyclictest0-21swapper/008:46:570
2672543992018,1cyclictest0-21swapper/008:43:460
2672543992018,1cyclictest0-21swapper/007:53:440
2672543992015,4cyclictest2789502-21seq12:36:190
267254399201,3cyclictest2768485-21munin-run11:40:000
2672543992012,6cyclictest2738043-21ntpq10:10:190
2672543992012,6cyclictest2725164-21expr09:35:140
2672543992011,7cyclictest883-21dbus-daemon09:30:010
267254399200,2cyclictest2764414-21python311:25:220
267254399200,1cyclictest0-21swapper/010:32:490
267254399200,1cyclictest0-21swapper/009:41:190
267254399200,19cyclictest2762419-21python311:20:210
267254399200,19cyclictest0-21swapper/010:20:190
267254399200,19cyclictest0-21swapper/009:16:310
267254399200,19cyclictest0-21swapper/008:11:010
267254399200,19cyclictest0-21swapper/007:39:310
139099200,14rtkit-daemon0-21swapper/212:08:272
139099200,14rtkit-daemon0-21swapper/211:33:462
139099200,13rtkit-daemon0-21swapper/208:34:132
27635482190,3sleep234-21ksoftirqd/211:25:112
2672556991917,1cyclictest3056-21PK-Backend09:05:353
2672556991912,5cyclictest2707746-21awk08:45:153
2672556991911,6cyclictest2749145-21cron10:45:003
267255699190,1cyclictest3056-21PK-Backend07:40:573
267255699190,1cyclictest0-21swapper/310:16:593
267255699190,18cyclictest3056-21PK-Backend10:00:573
267255699190,18cyclictest0-21swapper/308:21:203
2672552991917,1cyclictest0-21swapper/212:32:162
2672552991917,1cyclictest0-21swapper/211:59:442
2672552991917,1cyclictest0-21swapper/210:37:382
2672552991917,1cyclictest0-21swapper/208:06:582
2672552991917,1cyclictest0-21swapper/207:52:112
2672552991916,2cyclictest2781300-21tr12:15:132
2672552991916,2cyclictest0-21swapper/209:26:202
2672552991916,2cyclictest0-21swapper/208:24:082
2672552991916,2cyclictest0-21swapper/207:47:182
2672552991914,4cyclictest1980-21gvfs-udisks2-vo12:28:032
267255299190,1cyclictest0-21swapper/210:22:312
267255299190,1cyclictest0-21swapper/210:12:232
267255299190,1cyclictest0-21swapper/209:56:562
267255299190,1cyclictest0-21swapper/209:56:562
267255299190,1cyclictest0-21swapper/209:16:072
267255299190,18cyclictest0-21swapper/210:26:302
267255299190,18cyclictest0-21swapper/209:38:442
267254799192,11cyclictest2685466-21apt07:45:131
2672547991917,1cyclictest0-21swapper/109:42:421
2672547991912,5cyclictest2780854-21sed12:15:011
267254799191,13cyclictest1095-21runrttasks07:50:221
267254799190,1cyclictest3056-21PK-Backend10:39:411
267254799190,1cyclictest3056-21PK-Backend10:31:571
267254799190,1cyclictest3056-21PK-Backend10:12:091
267254799190,18cyclictest3056-21PK-Backend08:05:081
267254799190,18cyclictest0-21swapper/111:50:591
267254399194,13cyclictest2709406-21df_inode08:50:130
2672543991917,1cyclictest0-21swapper/012:06:520
2672543991917,1cyclictest0-21swapper/011:00:350
2672543991917,1cyclictest0-21swapper/010:44:590
2672543991917,1cyclictest0-21swapper/009:48:490
2672543991917,1cyclictest0-21swapper/008:28:450
2672543991917,1cyclictest0-21swapper/008:15:410
2672543991917,1cyclictest0-21swapper/007:24:300
2672543991915,3cyclictest2782612-21apt12:20:000
2672543991912,6cyclictest2769520-21pluginstate11:40:200
2672543991912,5cyclictest2735355-21cut10:05:000
267254399190,1cyclictest0-21swapper/009:14:010
267254399190,1cyclictest0-21swapper/008:03:110
267254399190,1cyclictest0-21swapper/007:47:490
267254399190,18cyclictest0-21swapper/012:32:070
267254399190,18cyclictest0-21swapper/012:25:310
267254399190,18cyclictest0-21swapper/009:57:070
267254399190,18cyclictest0-21swapper/009:57:070
267254399190,18cyclictest0-21swapper/009:22:050
267254399190,18cyclictest0-21swapper/008:05:270
267254399190,18cyclictest0-21swapper/007:44:590
267254399190,18cyclictest0-21swapper/007:32:500
139099190,13rtkit-daemon0-21swapper/212:21:592
2672556991816,1cyclictest0-21swapper/310:29:293
2672556991815,2cyclictest3056-21PK-Backend12:29:143
2672556991813,3cyclictest2760233-21idleruntime11:15:173
2672556991813,3cyclictest2756328-21/usr/sbin/munin11:05:123
2672556991812,4cyclictest2752808-21sed10:55:003
2672552991816,1cyclictest0-21swapper/212:04:582
2672552991816,1cyclictest0-21swapper/211:16:262
2672552991816,1cyclictest0-21swapper/210:54:482
2672552991816,1cyclictest0-21swapper/210:31:162
2672552991816,1cyclictest0-21swapper/210:00:072
2672552991816,1cyclictest0-21swapper/208:11:312
2672552991816,1cyclictest0-21swapper/208:04:412
2672552991816,1cyclictest0-21swapper/207:26:222
2672552991811,5cyclictest2682057-21sort07:35:122
267254799188,5cyclictest2701548-21apt-get08:30:031
267254799187,2cyclictest141rcu_preempt10:45:011
267254799182,13cyclictest2735683-21cpuspeed_turbos10:05:121
267254799182,12cyclictest2709785-21runrttasks08:50:191
2672547991816,1cyclictest3056-21PK-Backend10:28:111
2672547991816,1cyclictest3056-21PK-Backend07:23:051
2672547991816,1cyclictest3056-21PK-Backend07:15:261
2672547991816,1cyclictest0-21swapper/112:05:391
2672547991812,5cyclictest26-21ksoftirqd/111:40:221
2672543991816,1cyclictest0-21swapper/011:50:360
2672543991816,1cyclictest0-21swapper/009:03:070
2672543991816,1cyclictest0-21swapper/007:14:030
267254399181,15cyclictest2775188-21seq11:57:250
267254399180,9cyclictest918-21rs:main0
267254399180,1cyclictest0-21swapper/008:23:040
27497202170,4sleep13056-21PK-Backend10:45:121
2672556991715,1cyclictest3056-21PK-Backend12:03:253
267255699170,4cyclictest2753966-21users10:55:243
2672552991715,1cyclictest0-21swapper/209:43:172
2672552991715,1cyclictest0-21swapper/207:44:272
267254799179,3cyclictest26-21ksoftirqd/109:10:001
2672547991712,3cyclictest2783776-21switchtime12:20:221
2672543991713,3cyclictest13-21ksoftirqd/009:55:000
2672543991710,5cyclictest2766207-21cat11:30:210
2672543991710,5cyclictest2723871-21ntpq09:30:190
2672543991710,5cyclictest2711135-21df08:55:140
2672556991614,1cyclictest3056-21PK-Backend07:45:373
2672556991614,1cyclictest3056-21PK-Backend07:37:203
267255299168,6cyclictest2739558-21cat10:15:182
2672552991610,4cyclictest2673533-21awk07:10:202
267254399169,6cyclictest2757451-21sshd11:05:290
267254399169,5cyclictest2746686-21awk10:35:180
267254399161,11cyclictest2735563-21apt10:05:120
2672543991610,5cyclictest2702498-21ls08:30:220
267255699157,6cyclictest2717426-21cron09:15:003
2672543991513,1cyclictest0-21swapper/007:16:400
2672543991510,3cyclictest2758818-21grep11:10:190
267254799143,2cyclictest141rcu_preempt11:06:091
267254799141,3cyclictest3056-21PK-Backend07:10:201
2672547991410,3cyclictest3056-21PK-Backend11:14:361
2672547991410,3cyclictest3056-21PK-Backend10:16:331
2672543991412,1cyclictest0-21swapper/010:25:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional