You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-22 - 01:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Jun 21, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24279992625,1cyclictest34-21ksoftirqd/312:30:013
2426799261,24cyclictest0-21swapper/011:30:530
24279992524,1cyclictest34-21ksoftirqd/309:46:243
24279992523,1cyclictest0-21swapper/311:28:383
24277992523,1cyclictest8509-21munin-plugin-st11:05:002
24272992522,2cyclictest32210-21ssh09:54:201
2427299250,24cyclictest19373-21sh10:12:171
24267992522,2cyclictest30572-21rm10:23:410
2426799250,24cyclictest9810-21ssh11:36:370
2426799250,24cyclictest18465-21diskmemload10:38:040
24279992423,1cyclictest34-21ksoftirqd/311:09:503
2427999240,24cyclictest0-21swapper/311:11:553
2427999240,23cyclictest18465-21diskmemload10:00:453
2427999240,23cyclictest0-21swapper/308:15:173
2427999240,1cyclictest19301-21grep11:15:183
24277992420,3cyclictest0-21swapper/210:56:132
2427799240,24cyclictest0-21swapper/210:03:022
2427799240,23cyclictest9760-21rm11:36:332
2427799240,23cyclictest3949-21ssh09:57:362
2427799240,23cyclictest0-21swapper/211:51:312
2427799240,23cyclictest0-21swapper/211:26:462
2427299243,3cyclictest21697-21sh09:12:081
24272992424,0cyclictest18465-21diskmemload11:43:061
24272992423,1cyclictest18465-21diskmemload10:41:121
24272992422,1cyclictest0-21swapper/110:05:201
2427299240,24cyclictest0-21swapper/109:22:091
2427299240,23cyclictest19107-21apt-get11:15:171
2427299240,23cyclictest0-21swapper/111:20:091
2427299240,1cyclictest3456-21ssh10:28:271
2427299240,1cyclictest151rcu_preempt09:35:531
2427299240,1cyclictest0-21swapper/110:40:011
24267992424,0cyclictest0-21swapper/010:07:380
24267992423,1cyclictest8286-21ssh12:38:300
24267992423,1cyclictest0-21swapper/011:22:150
2426799241,1cyclictest15442-21sh12:14:480
2426799240,24cyclictest18465-21diskmemload10:52:490
2426799240,23cyclictest21711-21sh11:16:420
2426799240,23cyclictest18465-21diskmemload09:54:560
2426799240,23cyclictest0-21swapper/010:48:580
2426799240,1cyclictest18465-21diskmemload10:11:290
2426799240,1cyclictest0-21swapper/010:56:480
24279992323,0cyclictest18465-21diskmemload09:27:423
24279992323,0cyclictest0-21swapper/311:40:483
24279992321,1cyclictest0-21swapper/312:02:033
24279992321,1cyclictest0-21swapper/311:04:243
24279992321,1cyclictest0-21swapper/310:28:223
2427999231,21cyclictest34-21ksoftirqd/310:15:223
2427999230,2cyclictest16827-21ethtool12:15:193
2427999230,23cyclictest0-21swapper/310:56:013
2427999230,23cyclictest0-21swapper/309:10:113
2427999230,22cyclictest9347-21diskmemload09:30:543
2427999230,22cyclictest23766-21ssh12:21:223
2427999230,0cyclictest0-21swapper/312:37:203
2427999230,0cyclictest0-21swapper/309:56:173
24277992323,0cyclictest18465-21diskmemload09:50:262
24277992323,0cyclictest0-21swapper/209:11:142
24277992322,0cyclictest0-21swapper/210:45:182
24277992320,2cyclictest622-21gdbus07:40:012
24277992320,2cyclictest26510-21sh09:16:442
24277992320,2cyclictest11582-21sh11:06:592
2427799230,2cyclictest19502-21ssh10:44:152
2427799230,23cyclictest0-21swapper/212:01:372
2427799230,23cyclictest0-21swapper/209:41:492
2427799230,22cyclictest7377-21ssh12:36:572
2427799230,22cyclictest6816-21turbostat.cron11:35:012
2427799230,22cyclictest23848-21ssh12:21:422
2427799230,22cyclictest21609-21rm11:16:332
2427799230,22cyclictest20045-21ssh11:46:112
2427799230,22cyclictest18759-21ssh12:16:472
2427799230,22cyclictest18465-21diskmemload10:20:492
2427799230,0cyclictest18465-21diskmemload10:35:262
2427799230,0cyclictest18465-21diskmemload10:10:402
2427799230,0cyclictest0-21swapper/212:30:352
2427799230,0cyclictest0-21swapper/211:44:222
2427799230,0cyclictest0-21swapper/210:51:112
2427799230,0cyclictest0-21swapper/209:39:362
24272992323,0cyclictest18465-21diskmemload09:25:291
24272992323,0cyclictest0-21swapper/110:59:251
24272992323,0cyclictest0-21swapper/109:32:191
24272992323,0cyclictest0-21swapper/109:19:081
24272992322,1cyclictest6404-21ssh11:02:011
2427299230,2cyclictest16049-21sh12:15:151
2427299230,23cyclictest0-21swapper/112:35:341
2427299230,23cyclictest0-21swapper/112:24:121
2427299230,23cyclictest0-21swapper/111:57:221
2427299230,22cyclictest3275-21sh12:33:471
2427299230,22cyclictest31444-21cron07:25:011
2427299230,22cyclictest29566-21ssh12:27:391
2427299230,22cyclictest18465-21diskmemload10:03:121
2427299230,0cyclictest18465-21diskmemload10:33:191
2427299230,0cyclictest0-21swapper/111:52:411
2427299230,0cyclictest0-21swapper/110:54:531
24267992323,0cyclictest0-21swapper/012:18:260
24267992323,0cyclictest0-21swapper/011:44:290
24267992323,0cyclictest0-21swapper/011:27:490
24267992323,0cyclictest0-21swapper/010:03:110
2426799232,21cyclictest62550irq/127-eno111:51:080
24267992322,1cyclictest25826-21rm10:19:230
24267992322,1cyclictest0-21swapper/009:23:020
24267992321,1cyclictest14386-21diskstats11:10:180
24267992320,2cyclictest25777-21rm12:24:400
24267992320,2cyclictest19183-21idleruntime-cro08:05:000
2426799230,2cyclictest19356-21sh09:40:260
2426799230,23cyclictest0-21swapper/009:36:030
2426799230,23cyclictest0-21swapper/009:16:440
2426799230,22cyclictest606-21dbus-daemon08:30:000
2426799230,22cyclictest606-21dbus-daemon07:30:010
2426799230,22cyclictest5053-21ssh09:27:170
2426799230,22cyclictest27754-21readlink12:25:240
2426799230,22cyclictest0-21swapper/010:31:270
2426799230,1cyclictest0-21swapper/012:30:020
2426799230,0cyclictest18465-21diskmemload09:46:340
2426799230,0cyclictest0-21swapper/009:33:490
24279992222,0cyclictest18465-21diskmemload10:10:073
24279992222,0cyclictest0-21swapper/311:21:033
24279992222,0cyclictest0-21swapper/310:40:273
24279992221,0cyclictest341ksoftirqd/308:55:153
24279992220,1cyclictest2085-21apt-get11:30:153
24279992220,1cyclictest0-21swapper/312:30:073
24279992220,1cyclictest0-21swapper/311:56:163
24279992220,1cyclictest0-21swapper/311:45:223
24279992220,1cyclictest0-21swapper/311:35:043
2427999220,21cyclictest606-21dbus-daemon08:40:003
2427999220,21cyclictest5774-21ssh12:05:183
2427999220,21cyclictest331rcuc/309:40:083
2427999220,21cyclictest27953-21sh09:50:153
2427999220,21cyclictest18465-21diskmemload11:50:543
2427999220,21cyclictest18465-21diskmemload10:46:553
2427999220,21cyclictest14406-21ssh09:35:463
2427999220,21cyclictest0-21swapper/310:50:493
2427999220,21cyclictest0-21swapper/310:05:243
2427999220,21cyclictest0-21swapper/309:15:393
2427999220,1cyclictest6684-21ssh10:30:263
24277992222,0cyclictest18465-21diskmemload11:55:102
24277992222,0cyclictest0-21swapper/212:10:092
24277992222,0cyclictest0-21swapper/210:15:172
24277992220,1cyclictest0-21swapper/211:20:222
24277992220,1cyclictest0-21swapper/209:30:272
24277992218,3cyclictest9123-21dump-pmu-power07:45:002
2427799220,21cyclictest4979-21ssh09:27:102
2427799220,21cyclictest18465-21diskmemload10:25:182
2427799220,21cyclictest0-21swapper/212:06:252
2427799220,21cyclictest0-21swapper/211:13:042
2427799220,21cyclictest0-21swapper/207:25:152
2427799220,1cyclictest13104-21rm10:05:362
2427799220,1cyclictest0-21swapper/210:33:432
2427799220,1cyclictest0-21swapper/209:24:102
2427799220,0cyclictest18465-21diskmemload09:46:022
24272992222,0cyclictest0-21swapper/111:27:341
24272992222,0cyclictest0-21swapper/110:21:431
24272992222,0cyclictest0-21swapper/109:40:031
24272992221,1cyclictest14481-21ssh12:13:161
24272992220,1cyclictest1630-21awk08:35:001
2427299220,22cyclictest0-21swapper/111:47:331
2427299220,22cyclictest0-21swapper/111:05:291
2427299220,21cyclictest31719-21apt-get07:25:151
2427299220,21cyclictest26636-21latency_hist07:15:001
2427299220,21cyclictest14647-21apt-get07:55:141
2427299220,21cyclictest0-21swapper/111:11:081
2427299220,21cyclictest0-21swapper/108:30:001
2427299220,21cyclictest0-21swapper/108:00:161
2427299220,1cyclictest7755-21ssh12:05:391
2427299220,1cyclictest3093-21ssh12:01:261
2427299220,1cyclictest26394-21ls08:20:011
2427299220,1cyclictest24049-21sh10:16:401
2427299220,1cyclictest11141-21ssh11:38:421
2427299220,1cyclictest0-21swapper/111:34:301
2427299220,1cyclictest0-21swapper/109:48:121
2427299220,0cyclictest0-21swapper/110:46:431
24267992220,1cyclictest6339-21apt-get07:40:020
24267992220,1cyclictest595-21apt-get12:00:160
24267992220,1cyclictest20623-21ssh11:47:080
24267992220,1cyclictest11217-21cron08:55:000
24267992220,1cyclictest0-21swapper/011:05:520
24267992220,1cyclictest0-21swapper/010:42:390
2426799220,21cyclictest709-21apt-get09:55:150
2426799220,21cyclictest18741-21ssh09:10:040
2426799220,21cyclictest0-21swapper/011:55:110
2426799220,1cyclictest0-21swapper/012:05:150
2426799220,0cyclictest18465-21diskmemload11:01:440
2426799220,0cyclictest0-21swapper/010:28:060
24279992121,0cyclictest0-21swapper/307:50:173
24279992120,1cyclictest9105-21apt-get08:50:163
24279992120,1cyclictest0-21swapper/310:35:083
2427999210,20cyclictest0-21swapper/307:40:023
2427999210,20cyclictest0-21swapper/307:15:173
2427999210,1cyclictest313-21systemd-journal07:15:003
2427999210,0cyclictest0-21swapper/307:20:163
24277992120,1cyclictest29469-21apt-get08:25:152
24277992119,1cyclictest4038-21perf08:40:002
2427799210,21cyclictest26339-21apt-get12:25:172
24272992121,0cyclictest0-21swapper/109:05:171
24272992121,0cyclictest0-21swapper/108:20:171
2427299210,1cyclictest13971-21apt-get09:00:171
2427299210,1cyclictest0-21swapper/107:55:001
2427299210,0cyclictest0-21swapper/109:56:421
2426799210,21cyclictest0-21swapper/008:05:170
2426799210,20cyclictest0-21swapper/009:05:170
2426799210,20cyclictest0-21swapper/007:55:140
2426799210,0cyclictest0-21swapper/007:50:020
24279992020,0cyclictest0-21swapper/308:05:173
24279992019,0cyclictest0-21swapper/308:25:153
24279992018,1cyclictest0-21swapper/308:00:173
24277992020,0cyclictest6711-21apt-get08:45:152
24277992020,0cyclictest11500-21apt-get08:55:152
24277992020,0cyclictest0-21swapper/208:50:162
24277992020,0cyclictest0-21swapper/208:20:172
24277992020,0cyclictest0-21swapper/207:50:172
2427799200,20cyclictest0-21swapper/208:05:172
2427799200,0cyclictest0-21swapper/208:00:162
24272992020,0cyclictest0-21swapper/107:40:021
24272992020,0cyclictest0-21swapper/107:35:161
24272992020,0cyclictest0-21swapper/107:15:171
24272992018,1cyclictest11539-21munin-plugin-st07:50:001
24267992020,0cyclictest26904-21apt-get07:15:170
24267992020,0cyclictest0-21swapper/008:45:150
24267992020,0cyclictest0-21swapper/008:40:150
24267992020,0cyclictest0-21swapper/007:45:140
24267992020,0cyclictest0-21swapper/007:20:160
24279991918,1cyclictest0-21swapper/309:00:173
24279991917,1cyclictest18486-21grep09:10:013
2427999190,18cyclictest4101-21apt-get07:35:163
24277991919,0cyclictest0-21swapper/207:45:142
24277991918,0cyclictest31916-21apt-get08:30:162
24272991919,0cyclictest0-21swapper/108:50:161
24272991919,0cyclictest0-21swapper/107:30:161
24267991919,0cyclictest0-21swapper/008:55:150
24267991917,1cyclictest0-21swapper/008:25:010
2426799190,10cyclictest0-21swapper/007:30:160
24279991818,0cyclictest0-21swapper/308:10:153
24279991816,1cyclictest3822-21turbostat07:35:013
24277991816,1cyclictest21852-21apt-get08:10:162
24272991818,0cyclictest4316-21apt-get08:40:151
24272991818,0cyclictest0-21swapper/108:45:151
24267991817,0cyclictest0-21swapper/008:15:170
24279991716,1cyclictest34-21ksoftirqd/308:25:013
24277991714,2cyclictest31460-21awk07:25:012
24277991714,2cyclictest18458-21cron09:10:012
24267991717,0cyclictest0-21swapper/008:35:170
24267991714,2cyclictest1603-21idleruntime-cro08:35:000
24279991616,0cyclictest0-21swapper/308:30:163
24277991616,0cyclictest0-21swapper/208:45:002
24277991616,0cyclictest0-21swapper/208:15:162
2427799160,15cyclictest25822-21sendmail_mailqu07:10:232
2427799160,15cyclictest0-21swapper/207:55:162
2427999151,7cyclictest0-21swapper/308:45:003
24279991513,1cyclictest16780-21turbostat08:00:003
24277991515,0cyclictest0-21swapper/209:00:172
24277991513,1cyclictest1701-21apt-get07:30:162
24272991513,1cyclictest4056-21awk08:40:001
24272991513,1cyclictest0-21swapper/108:55:241
24267991515,0cyclictest21852-21apt-get08:10:150
2426799150,7cyclictest622-21gdbus09:05:010
2426799150,1cyclictest0-21swapper/007:10:180
24279991414,0cyclictest0-21swapper/308:45:373
2427999140,0cyclictest0-21swapper/307:45:173
2427999140,0cyclictest0-21swapper/307:25:183
2427799140,0cyclictest0-21swapper/207:16:002
24272991414,0cyclictest0-21swapper/108:06:251
24267991414,0cyclictest0-21swapper/007:35:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional