You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 19:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Dec 19, 2025 12:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10439276325,694sleep110485-21unin-run09:24:381
10439276325,694sleep110485-21unin-run09:24:381
22160261512,508sleep122311-21head08:15:081
2843125966,579sleep128475-21ntpq10:20:021
2843125966,579sleep128475-21ntpq10:20:021
11489254110,409sleep111545-21ls11:10:021
11489254110,409sleep111545-21ls11:10:021
134632477134,192sleep113527-21latency07:50:001
31434245413,423sleep131583-21tail08:45:171
59552411161,209sleep16030-21sendmail_mailtr07:25:081
59552411161,209sleep16030-21sendmail_mailtr07:25:081
9220238625,299sleep19380-21awk07:35:101
2001823186,260sleep119358-21kworker/u8:008:10:111
2001823186,260sleep119358-21kworker/u8:008:10:111
18284226517,223sleep318375-21latency_hist09:49:393
20975225912,231sleep121062-21grep11:39:581
23125225155,171sleep123154-21fschecks_time08:20:011
14199222578,108sleep114291-21sshd11:19:391
2321022179,195sleep123233-21fschecks_count10:04:561
2321022179,195sleep123233-21fschecks_count10:04:561
246122129,186sleep02500-21ntpq07:15:020
21579219919,151sleep10-21swapper/109:59:511
21579219919,151sleep10-21swapper/109:59:511
41321979,172sleep1479-21wc07:10:061
41321979,172sleep1479-21wc07:10:061
15144219718,150sleep10-21swapper/109:39:401
15144219718,150sleep10-21swapper/109:39:401
1425219614,157sleep10-21swapper/112:20:071
6531219113,120sleep16593-21uname09:10:081
644218816,155sleep1699-21latency_hist12:19:381
8864217614,147sleep18926-21modprobe09:19:371
459021708,147sleep14721-21timerwakeupswit12:30:081
2503621696,16sleep10-21swapper/110:10:011
1366216816,44sleep10-21swapper/108:54:501
1366216816,44sleep10-21swapper/108:54:501
10037216313,58sleep10-21swapper/111:05:011
1702321619,112sleep10-21swapper/109:45:051
5213215812,91sleep10-21swapper/107:24:401
5213215812,91sleep10-21swapper/107:24:401
1373821588,13sleep00-21swapper/007:50:070
2771521549,109sleep11-21systemd10:19:381
2771521549,109sleep11-21systemd10:19:381
1550721535,29sleep10-21swapper/111:22:131
16633215113,101sleep116654-21idleruntime-cro09:44:371
16633215113,101sleep116654-21idleruntime-cro09:44:371
1940821499,128sleep0369-21snmpd09:50:330
3272821487,32sleep10-21swapper/108:50:051
3272821487,32sleep10-21swapper/108:50:051
20611214510,97sleep120695-21wc09:55:071
25817214311,80sleep10-21swapper/111:55:001
1692021437,44sleep10-21swapper/108:00:501
1692021437,44sleep10-21swapper/108:00:501
686921425,11sleep10-21swapper/112:37:471
686921425,11sleep10-21swapper/112:37:471
21260214130,88sleep321266-21modprobe09:59:373
12659214014,58sleep10-21swapper/109:30:151
25097213715,106sleep324517-21/usr/sbin/munin08:25:073
32263213620,88sleep10-21swapper/107:09:381
30312213510,109sleep030417-21fschecks_time12:09:560
30312213510,109sleep030417-21fschecks_time12:09:560
7984213398,17sleep10-21swapper/110:59:401
3061121338,65sleep130646-21ntpq12:10:011
3061121338,65sleep130646-21ntpq12:10:011
15976213311,105sleep00-21swapper/011:24:510
30752132100,19sleep13065-21munin-run10:44:381
1140921308,87sleep111408-21munin-run07:44:391
1140921308,87sleep111408-21munin-run07:44:391
27224212917,52sleep10-21swapper/108:33:091
1299821297,83sleep113151-21date07:49:401
633321247,106sleep1369-21snmpd10:54:191
5160212218,86sleep25283-21timerwakeupswit09:05:082
2896021229,17sleep10-21swapper/108:39:391
6995212110,17sleep10-21swapper/107:29:531
2807921215,12sleep10-21swapper/112:00:411
2807921215,12sleep10-21swapper/112:00:411
2376721219,97sleep123877-21sed11:49:391
11207212116,19sleep00-21swapper/009:25:210
2770021164,100sleep00-21swapper/010:19:290
2770021164,100sleep00-21swapper/010:19:290
888211436,48sleep2884-21apt-get12:19:512
3537211415,83sleep00-21swapper/010:44:490
29929211412,45sleep10-21swapper/112:09:371
29929211412,45sleep10-21swapper/112:09:371
250221147,14sleep10-21swapper/107:15:031
24726211323,71sleep324866-21grep11:50:093
4047211210,89sleep14081-21ps10:45:221
29039211121,72sleep20-21swapper/212:05:022
29039211121,72sleep20-21swapper/212:05:022
18902211116,76sleep319001-21latency_hist11:34:393
16380211113,37sleep10-21swapper/111:25:041
476421107,13sleep10-21swapper/109:04:591
29781211028,70sleep129823-21if_eth010:24:591
29781211028,70sleep129823-21if_eth010:24:591
1814721078,16sleep10-21swapper/108:04:541
1814721078,16sleep10-21swapper/108:04:541
3156921067,13sleep10-21swapper/110:30:041
3156921067,13sleep10-21swapper/110:30:041
1932821066,44sleep10-21swapper/111:35:101
1932821066,44sleep10-21swapper/111:35:101
1499210512,70sleep31539-21munin-node07:14:403
1499210512,70sleep31539-21munin-node07:14:403
12983210517,75sleep213063-21fschecks_time11:14:592
5283210420,26sleep30-21swapper/312:34:383
240421047,15sleep10-21swapper/112:24:471
1806921047,15sleep10-21swapper/111:30:091
88421036,80sleep00-21swapper/010:35:080
88421036,80sleep00-21swapper/010:35:080
18556210322,62sleep318645-21tune2fs09:49:563
14521210319,63sleep214662-21unixbench_multi09:35:212
14521210319,63sleep214662-21unixbench_multi09:35:212
3266721027,35sleep10-21swapper/110:34:521
3266721027,35sleep10-21swapper/110:34:521
5846210120,62sleep35992-21unixbench_singl10:50:093
1226721010,12sleep20-21swapper/207:45:072
1121421018,37sleep132369-21cyclictest09:25:231
848121007,54sleep10-21swapper/111:00:001
21332210011,17sleep00-21swapper/011:40:050
144229919,62sleep21497-21latency_hist07:14:392
144229919,62sleep21497-21latency_hist07:14:392
308229810,24sleep10-21swapper/108:59:581
184652989,48sleep10-21swapper/109:49:501
1598229824,37sleep20-21swapper/211:24:532
32432970,10sleep33157-21apt-key10:44:393
245452966,48sleep10-21swapper/108:24:531
159162966,57sleep1369-21snmpd07:58:441
159162966,57sleep1369-21snmpd07:58:441
1536129514,58sleep015480-21cat07:55:080
1536129514,58sleep015480-21cat07:55:080
574829320,58sleep35753-21modprobe12:34:503
574829320,58sleep35753-21modprobe12:34:503
3272429312,67sleep232726-21sshd12:15:482
240092937,47sleep10-21swapper/111:49:511
1927229313,66sleep3228-21systemd-journal11:34:473
1927229313,66sleep3228-21systemd-journal11:34:473
3229229218,61sleep232299-21cpu08:49:542
3229229218,61sleep232299-21cpu08:49:542
2965129210,16sleep20-21swapper/210:24:562
2965129210,16sleep20-21swapper/210:24:562
97462908,56sleep00-21swapper/011:04:480
341829012,63sleep03426-21ntpq09:00:060
2800829015,60sleep328052-21ntpq08:35:053
2212329017,57sleep322144-21awk11:44:363
2953328939,33sleep10-21swapper/108:40:041
2953328939,33sleep10-21swapper/108:40:041
1573828816,53sleep015787-21latency_hist11:24:380
2664928714,49sleep326420-21/usr/sbin/munin10:15:123
2664928714,49sleep326420-21/usr/sbin/munin10:15:123
1352228716,50sleep313552-21basename09:34:373
1144228718,18sleep00-21swapper/011:10:010
1144228718,18sleep00-21swapper/011:10:010
396228610,18sleep30-21swapper/307:20:343
396228610,18sleep30-21swapper/307:20:343
3203628621,15sleep30-21swapper/308:49:393
63502850,8sleep00-21swapper/009:10:020
1855828514,52sleep018074-21/usr/sbin/munin08:05:050
1855828514,52sleep018074-21/usr/sbin/munin08:05:050
368228426,35sleep20-21swapper/212:29:362
175928410,59sleep21865-21cpu07:14:432
1101228417,53sleep211024-21vmstat07:40:162
1101228417,53sleep211024-21vmstat07:40:162
955228315,55sleep29598-21/usr/sbin/munin09:20:022
955228315,55sleep29598-21/usr/sbin/munin09:20:022
767328312,56sleep27682-21/usr/sbin/munin09:14:542
580728317,46sleep25742-21apt-get12:35:302
580728317,46sleep25742-21apt-get12:35:302
522328321,42sleep25281-21systemd-run12:34:372
3098428311,56sleep031005-21apt-get07:04:490
2225028310,57sleep222358-21cut11:44:392
1101328314,20sleep30-21swapper/309:25:163
315852820,7sleep20-21swapper/208:45:172
242072827,63sleep00-21swapper/008:24:030
2584628127,24sleep20-21swapper/208:29:392
2392128113,49sleep224024-21users10:05:092
2392128113,49sleep224024-21users10:05:092
2368728117,41sleep223832-21sed08:20:142
834428013,54sleep38416-21fschecks_time10:59:583
894427913,52sleep39054-21latency_hist09:19:393
871827917,43sleep08797-21/usr/sbin/munin11:00:070
270427915,30sleep00-21swapper/012:24:580
1900627921,42sleep019180-21apt-config11:34:400
2555027822,34sleep225544-21apt-get11:54:532
1708627821,40sleep217150-21ps08:00:552
1708627821,40sleep217150-21ps08:00:552
643027712,50sleep06542-21cat12:35:520
643027712,50sleep06542-21cat12:35:520
3151727712,15sleep20-21swapper/212:14:382
3151727712,15sleep20-21swapper/212:14:382
1201427724,25sleep30-21swapper/311:10:203
1201427724,25sleep30-21swapper/311:10:203
972627612,26sleep30-21swapper/309:20:083
972627612,26sleep30-21swapper/309:20:083
1739827618,41sleep217422-21/usr/sbin/munin09:45:142
591727511,48sleep00-21swapper/007:25:070
591727511,48sleep00-21swapper/007:25:070
421827523,30sleep34252-21munin-run09:04:373
2562027512,48sleep025627-21df_inode11:54:550
2026327517,43sleep220264-21sshd11:35:472
2026327517,43sleep220264-21sshd11:35:472
1551927518,29sleep30-21swapper/309:40:113
1551927518,29sleep30-21swapper/309:40:113
1471127528,25sleep30-21swapper/311:20:013
3137627414,40sleep031391-21kernelversion10:29:590
3137627414,40sleep031391-21kernelversion10:29:590
2370527415,45sleep3369-21snmpd11:49:293
765627323,29sleep30-21swapper/307:30:113
356527223,26sleep30-21swapper/310:44:523
2350427225,23sleep30-21swapper/308:20:113
787527116,22sleep00-21swapper/009:15:000
396427111,41sleep04014-21cpuspeed07:20:350
396427111,41sleep04014-21cpuspeed07:20:350
3128127115,29sleep30-21swapper/307:05:003
2442827124,24sleep30-21swapper/310:09:383
2442827124,24sleep30-21swapper/310:09:383
1727127123,28sleep30-21swapper/311:29:383
335127022,32sleep33374-21munin-run07:19:373
2278627020,26sleep00-21swapper/011:45:020
2001927010,45sleep220094-21fschecks_time08:10:122
2001927010,45sleep220094-21fschecks_time08:10:122
1184027026,27sleep00-21swapper/007:44:580
1048927015,22sleep00-21swapper/007:40:060
1048927015,22sleep00-21swapper/007:40:060
3211726917,36sleep332126-21needreboot12:15:003
2994726926,23sleep30-21swapper/308:40:143
2994726926,23sleep30-21swapper/308:40:143
1606626919,26sleep30-21swapper/307:59:393
1606626919,26sleep30-21swapper/307:59:393
715026824,19sleep00-21swapper/010:55:070
572126815,36sleep25845-21sed10:50:082
2597726818,27sleep20-21swapper/210:14:382
2448626815,29sleep00-21swapper/011:50:060
242926823,28sleep32430-21sshd12:24:493
1512926811,31sleep20-21swapper/207:55:022
1512926811,31sleep20-21swapper/207:55:022
1408226813,36sleep314097-21/usr/sbin/munin07:50:103
53842679,35sleep00-21swapper/012:34:390
305612675,22sleep00-21swapper/010:26:110
305612675,22sleep00-21swapper/010:26:110
1267426720,23sleep30-21swapper/311:14:403
978926615,34sleep09834-21systemctl07:39:370
2461926624,22sleep20-21swapper/211:50:082
2231326613,23sleep20-21swapper/208:15:092
13232664,50sleep00-21swapper/007:13:450
13232664,50sleep00-21swapper/007:13:450
979726524,19sleep00-21swapper/009:20:090
979726524,19sleep00-21swapper/009:20:090
2900026511,38sleep229067-21date08:39:402
2849626522,24sleep00-21swapper/010:20:050
2849626522,24sleep00-21swapper/010:20:050
301392645,11sleep20-21swapper/208:40:332
301392645,11sleep20-21swapper/208:40:332
2692626413,28sleep20-21swapper/210:15:192
2692626413,28sleep20-21swapper/210:15:192
2000926417,25sleep20-21swapper/209:54:512
1833326414,31sleep218367-21irqstats08:04:592
1833326414,31sleep218367-21irqstats08:04:592
126126420,28sleep00-21swapper/012:20:010
666726313,24sleep30-21swapper/309:10:103
2015726312,32sleep020170-21irqstats08:10:140
2015726312,32sleep020170-21irqstats08:10:140
71026217,28sleep2787-21awk10:35:062
71026217,28sleep2787-21awk10:35:062
336126215,26sleep20-21swapper/209:00:042
3018526219,22sleep30-21swapper/312:09:473
3018526219,22sleep30-21swapper/312:09:473
2221926220,22sleep00-21swapper/010:00:060
2221926220,22sleep00-21swapper/010:00:060
1576026222,21sleep00-21swapper/009:40:160
1576026222,21sleep00-21swapper/009:40:160
1506526216,21sleep30-21swapper/309:39:383
1506526216,21sleep30-21swapper/309:39:383
127062620,8sleep20-21swapper/209:30:152
3266026113,28sleep00-21swapper/012:15:090
2462826121,21sleep00-21swapper/008:24:560
2299526118,19sleep20-21swapper/211:45:082
194826115,22sleep30-21swapper/308:55:123
194826115,22sleep30-21swapper/308:55:123
1659226115,31sleep316650-21cpuspeed08:00:283
1659226115,31sleep316650-21cpuspeed08:00:283
1658926116,27sleep016533-21apt-get08:00:250
1658926116,27sleep016533-21apt-get08:00:250
1243326124,21sleep00-21swapper/009:30:100
675426018,20sleep30-21swapper/307:29:393
675426018,20sleep30-21swapper/307:29:393
3269726014,30sleep032727-21/usr/sbin/munin08:50:040
3269726014,30sleep032727-21/usr/sbin/munin08:50:040
2984526017,25sleep329939-21memory10:25:013
2984526017,25sleep329939-21memory10:25:013
2719926012,32sleep027206-21diskstats11:59:550
2719926012,32sleep027206-21diskstats11:59:550
2651326017,27sleep226537-21ntp_states08:30:032
264126019,23sleep02670-21ls08:59:370
264126019,23sleep02670-21ls08:59:370
219672606,12sleep20-21swapper/210:00:002
219672606,12sleep20-21swapper/210:00:002
214626021,22sleep00-21swapper/010:40:000
1817626013,29sleep018207-21munin-run09:49:360
3231025917,22sleep20-21swapper/207:09:392
2338625917,20sleep00-21swapper/010:04:590
2338625917,20sleep00-21swapper/010:04:590
1066625915,22sleep30-21swapper/307:40:133
1066625915,22sleep30-21swapper/307:40:133
701525811,23sleep30-21swapper/310:55:013
2493325811,27sleep30-21swapper/310:09:593
1127825815,21sleep20-21swapper/209:25:242
1106625815,25sleep211076-21munin-run11:09:382
839225714,29sleep08448-21sed07:34:400
470625711,24sleep00-21swapper/009:04:580
225362575,17sleep00-21swapper/008:17:450
2029525717,17sleep30-21swapper/308:10:173
2029525717,17sleep30-21swapper/308:10:173
3242225611,31sleep332465-21date10:34:383
3242225611,31sleep332465-21date10:34:383
2992125616,24sleep029928-21modprobe12:09:360
2992125616,24sleep029928-21modprobe12:09:360
271662568,14sleep00-21swapper/008:32:230
2555125617,24sleep325618-21df11:54:543
1873825616,18sleep30-21swapper/308:05:083
1873825616,18sleep30-21swapper/308:05:083
1813725617,18sleep20-21swapper/211:30:102
1343725619,22sleep00-21swapper/011:15:100
692125516,22sleep27014-21memory10:55:002
423125513,24sleep24249-21irqstats07:20:532
423125513,24sleep24249-21irqstats07:20:532
2906825516,19sleep30-21swapper/312:05:033
2906825516,19sleep30-21swapper/312:05:033
2847725516,21sleep20-21swapper/210:20:032
2847725516,21sleep20-21swapper/210:20:032
1960225513,23sleep00-21swapper/011:35:210
1960225513,23sleep00-21swapper/011:35:210
1908125513,18sleep20-21swapper/209:50:072
154525521,20sleep30-21swapper/310:39:393
154525521,20sleep30-21swapper/310:39:393
1174425519,19sleep30-21swapper/307:44:573
699625413,24sleep26985-21apt-get07:29:542
498625419,21sleep25019-21latency_hist10:49:432
177425416,25sleep21822-21ntpq08:55:072
177425416,25sleep21822-21ntpq08:55:072
977225312,24sleep39765-21apt-get11:04:513
731125315,19sleep20-21swapper/209:14:372
639825312,25sleep06408-21munin-run10:54:380
460525319,18sleep30-21swapper/309:04:573
2842925314,20sleep30-21swapper/312:04:383
2842925314,20sleep30-21swapper/312:04:383
2800325317,19sleep30-21swapper/310:19:483
2800325317,19sleep30-21swapper/310:19:483
2762425314,23sleep027621-21apt-get08:34:500
224282534,37sleep30-21swapper/308:16:273
212142535,36sleep00-21swapper/009:59:010
97662524,39sleep30-21swapper/307:39:203
233725214,24sleep22409-21sort10:40:062
849625111,21sleep28590-21memory11:00:012
2927125114,23sleep029278-21df_inode08:39:540
2927125114,23sleep029278-21df_inode08:39:540
2466825112,23sleep024660-21apt-get10:09:520
589525011,22sleep25906-21ntp_states07:25:052
589525011,22sleep25906-21ntp_states07:25:052
310962507,17sleep20-21swapper/210:29:492
310962507,17sleep20-21swapper/210:29:492
1529925012,22sleep215345-21users11:20:122
2200524914,20sleep322029-21ntp_states10:00:013
2200524914,20sleep322029-21ntp_states10:00:013
1262224816,17sleep20-21swapper/211:14:392
1262224816,17sleep20-21swapper/211:14:392
95932476,31sleep20-21swapper/207:37:052
3120124711,16sleep00-21swapper/008:45:110
271142478,15sleep20-21swapper/211:59:492
271142478,15sleep20-21swapper/211:59:492
1328024711,20sleep213375-21fschecks_time07:49:572
323002468,14sleep30-21swapper/308:49:553
323002468,14sleep30-21swapper/308:49:553
2605724618,15sleep30-21swapper/308:29:483
214024611,20sleep32159-21munin-run12:24:373
1436224611,15sleep00-21swapper/009:35:190
1436224611,15sleep00-21swapper/009:35:190
1665624111,17sleep20-21swapper/209:44:382
1665624111,17sleep20-21swapper/209:44:382
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional