You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 23:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Dec 30, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21069271413,593sleep021207-21unixbench_singl07:59:330
21069271413,593sleep021207-21unixbench_singl07:59:330
359124360,32sleep23595-21unixbench_singl10:29:322
1091323810,13sleep00-21swapper/007:29:030
2586523559,285sleep025867-21cpu09:59:150
12194232821,285sleep112264-21grep09:14:301
26514231820,285sleep126534-21uname11:44:231
29731229713,270sleep129769-21ntpq11:54:241
29731229713,270sleep129769-21ntpq11:54:241
25441228720,250sleep225465-21/usr/sbin/munin08:14:292
12021228210,258sleep012062-21ntpq09:14:250
175622242110,114sleep017570-21modprobe09:33:590
13863223919,205sleep313931-21ls11:04:223
13863223919,205sleep313931-21ls11:04:223
43592237114,75sleep04362-21cron12:18:590
7146223630,141sleep07211-21fschecks_time08:59:390
31859223120,147sleep032013-21latency_hist10:19:020
2850222741,164sleep02976-21latency_hist12:14:020
16134222518,192sleep116280-21awk11:10:221
16134222518,192sleep116280-21awk11:10:221
11490222038,64sleep00-21swapper/007:29:280
23024221838,119sleep00-21swapper/009:49:300
1126322150,6sleep20-21swapper/210:54:312
9719221415,162sleep00-21swapper/009:07:540
13681221192,74sleep013750-21fschecks_time11:04:190
13681221192,74sleep013750-21fschecks_time11:04:190
8112221011,186sleep08249-21grep10:44:330
20727220213,152sleep020852-21grep09:44:010
20727220213,152sleep020852-21grep09:44:010
32350220185,60sleep00-21swapper/010:19:220
1918321979,68sleep019309-21sed11:19:340
18657217520,14sleep00-21swapper/009:34:320
18657217520,14sleep00-21swapper/009:34:320
841521749,152sleep08455-21uname07:19:310
15443216517,134sleep315507-21cat07:44:013
24688216233,111sleep024689-21cron08:13:590
3334216036,98sleep00-21swapper/010:29:310
15716215640,96sleep00-21swapper/011:10:130
15716215640,96sleep00-21swapper/011:10:130
17954215510,48sleep00-21swapper/007:49:310
20132215314,118sleep020153-21fschecks_count11:24:180
593021518,22sleep00-21swapper/010:39:200
13350215110,125sleep013434-21perl09:19:210
1160214716,16sleep00-21swapper/008:39:310
1160214716,16sleep00-21swapper/008:39:310
25085214511,74sleep00-21swapper/011:39:250
3143421446,122sleep00-21swapper/008:34:180
24261214429,90sleep00-21swapper/009:54:110
28943214222,14sleep00-21swapper/010:09:130
12941214111,116sleep112977-21ntpq07:34:261
2250021409,116sleep022573-21xargs08:04:340
7731213640,79sleep07871-21latency_hist07:19:020
1289721367,15sleep00-21swapper/007:34:250
4197213111,48sleep00-21swapper/008:49:330
26925213115,95sleep00-21swapper/008:19:240
22042213111,108sleep022046-21ntpq11:29:270
26754213036,77sleep026837-21mailstats11:44:300
5520212945,56sleep35550-21awk10:38:593
3081021285,12sleep00-21swapper/008:32:530
323321276,36sleep00-21swapper/008:47:350
21054212727,79sleep121095-21fschecks_count09:44:181
6225212622,17sleep00-21swapper/012:24:160
5946212625,16sleep00-21swapper/012:23:590
3050221265,11sleep00-21swapper/011:57:500
3050221265,11sleep00-21swapper/011:57:500
23117212612,99sleep123191-21find09:49:311
83121230,27sleep1835-21unixbench_multi12:04:331
5484212359,45sleep00-21swapper/010:38:320
634921226,33sleep00-21swapper/008:57:490
634921226,33sleep00-21swapper/008:57:490
2424621205,32sleep00-21swapper/011:37:070
889121185,11sleep00-21swapper/012:32:190
25246211819,17sleep00-21swapper/008:14:230
32054211711,86sleep132073-21apt-get10:19:111
10567211734,66sleep010077-21/usr/sbin/munin09:09:280
5737211620,76sleep15836-21latency_hist07:14:021
3458211527,69sleep13586-21unixbench_singl10:29:321
27042211523,17sleep10-21swapper/108:19:291
2423211512,88sleep02433-21modprobe10:28:590
7923211416,76sleep17935-21vmstat08:59:521
17770211216,78sleep217836-21swap07:49:292
13179211219,69sleep313286-21timerandwakeup07:34:323
2247421117,93sleep2197-21jbd2/mmcblk2p2-09:49:042
12386211120,71sleep312535-21grep09:14:323
2157021087,87sleep021645-21xargs09:44:290
17324210712,77sleep317371-21/usr/sbin/munin11:14:283
6195210628,26sleep16230-21cat07:14:061
1568121068,35sleep00-21swapper/007:44:140
8033210517,65sleep38072-21cut07:19:213
32729210515,74sleep3360-21load12:04:243
32216210230,58sleep0369-21snmpd12:04:000
30003210215,71sleep330126-21timerwakeupswit11:54:303
30003210215,71sleep330126-21timerwakeupswit11:54:303
1578021025,85sleep00-21swapper/009:25:520
2871021016,40sleep028720-21munin-run10:09:000
25066210123,51sleep125105-21cut08:14:201
162429813,71sleep31635-21ntpq10:24:303
863729714,70sleep28671-21awk10:48:592
2438729727,54sleep224418-21modprobe11:38:582
1953629715,16sleep30-21swapper/307:54:313
181032965,58sleep00-21swapper/011:17:560
147132956,30sleep00-21swapper/007:39:300
95762937,74sleep09665-21cat12:34:210
748229226,41sleep37588-21tune2fs10:44:203
1951229222,50sleep319575-21fschecks_time09:39:193
1951229222,50sleep319575-21fschecks_time09:39:193
43892906,29sleep00-21swapper/007:09:090
1701829026,32sleep30-21swapper/307:49:023
104442906,28sleep00-21swapper/010:54:100
99632885,30sleep00-21swapper/010:51:070
293292885,29sleep00-21swapper/008:28:410
1180328814,59sleep311876-21cat10:59:013
2010328716,53sleep120041-21latency_hist07:59:021
130822875,28sleep00-21swapper/011:01:040
289772866,28sleep00-21swapper/011:53:060
895328416,49sleep38998-21/usr/sbin/munin09:04:263
1491728420,38sleep10-21swapper/111:09:011
1491728420,38sleep10-21swapper/111:09:011
1233228414,55sleep10-21swapper/107:34:021
2671228315,48sleep326222-21/usr/sbin/munin11:44:293
292128231,33sleep32945-21fschecks_count07:04:203
837128125,31sleep10-21swapper/112:29:301
2340628120,35sleep10-21swapper/111:34:211
211732806,16sleep20-21swapper/211:27:192
2692327919,39sleep227048-21timerwakeupswit11:44:322
2433327919,34sleep224280-21apt-get09:54:412
924027819,42sleep19388-21grep09:04:321
1922127812,49sleep219227-21ntp_states07:54:262
104727718,46sleep00-21swapper/012:06:150
3216427610,45sleep232180-21apt12:03:582
2719827615,46sleep327316-21timerwakeupswit08:19:313
2532027617,42sleep225440-21timerandwakeup11:39:312
1631427617,45sleep216382-21cpuspeed09:29:172
1331327616,48sleep313348-21fschecks_count09:19:203
282542755,56sleep10-21swapper/111:49:281
2186227512,48sleep321921-21/usr/sbin/munin11:29:223
1079327522,27sleep30-21swapper/307:29:033
939227413,29sleep20-21swapper/209:04:332
487427423,34sleep24972-21sed10:34:302
400227427,22sleep20-21swapper/208:49:272
284092748,12sleep20-21swapper/208:24:232
2473927419,37sleep20-21swapper/208:14:012
2361527414,40sleep323677-21/usr/sbin/munin11:34:273
199062746,12sleep00-21swapper/007:57:310
103027411,11sleep20-21swapper/208:39:292
103027411,11sleep20-21swapper/208:39:292
3207727311,44sleep30-21swapper/310:19:133
3156427319,30sleep231610-21fschecks_time08:34:332
3076727320,29sleep10-21swapper/110:14:221
2896027312,29sleep30-21swapper/310:09:163
1749627318,34sleep30-21swapper/307:49:223
568527214,36sleep25762-21awk08:54:282
568527214,36sleep25762-21awk08:54:282
3049727212,46sleep230544-21users08:29:322
283827226,23sleep00-21swapper/007:04:170
154727215,42sleep21541-21apt-get12:09:152
646227123,31sleep26554-21sort07:14:202
365027122,22sleep30-21swapper/312:14:293
774027012,38sleep27759-21apt-get12:29:102
454627012,39sleep34558-21vmstat08:49:363
2855227014,16sleep10-21swapper/108:24:261
1969527020,33sleep219764-21latency09:39:222
1969527020,33sleep219764-21latency09:39:222
1526127014,22sleep30-21swapper/309:24:293
2546626921,29sleep30-21swapper/308:14:303
1501626916,28sleep20-21swapper/211:09:022
1501626916,28sleep20-21swapper/211:09:022
1135426913,41sleep311390-21idleruntime-cro09:13:593
629526823,27sleep30-21swapper/310:39:273
567126823,28sleep25678-21modprobe07:13:592
2946526819,34sleep129543-21mailstats10:09:301
1311126818,16sleep20-21swapper/207:34:312
1228026815,37sleep312298-21awk07:33:593
67362676,46sleep30-21swapper/307:14:253
3276126718,32sleep3449-21latency_hist08:39:023
2384326719,23sleep30-21swapper/309:54:013
2229926714,34sleep322345-21/usr/sbin/munin08:04:293
1910426721,28sleep319128-21munin-run09:38:593
1910426721,28sleep319128-21munin-run09:38:593
1858426714,28sleep30-21swapper/311:19:213
1543026727,19sleep10-21swapper/109:24:311
948326618,24sleep10-21swapper/107:24:121
690126620,23sleep30-21swapper/312:24:313
3183326618,22sleep30-21swapper/310:19:013
2866926622,28sleep328695-21munin-run10:08:593
2289626622,21sleep10-21swapper/111:34:011
1638326612,38sleep316390-21df_inode09:29:183
1414126613,38sleep214191-21/usr/sbin/munin07:39:162
54326521,24sleep10-21swapper/108:39:161
54326521,24sleep10-21swapper/108:39:161
467826524,23sleep10-21swapper/112:19:161
3171626522,19sleep10-21swapper/108:34:361
2835326523,21sleep30-21swapper/311:49:303
1191526520,23sleep20-21swapper/209:14:232
832526423,19sleep30-21swapper/309:04:103
715226421,21sleep10-21swapper/110:44:021
2909526416,22sleep20-21swapper/211:54:012
2228726431,16sleep20-21swapper/211:29:332
901826318,27sleep29101-21fschecks_time10:49:202
555126314,20sleep10-21swapper/108:54:231
555126314,20sleep10-21swapper/108:54:231
480226315,21sleep10-21swapper/110:34:291
30082637,12sleep20-21swapper/208:44:382
1862726319,30sleep118708-21grep11:19:231
166926314,26sleep30-21swapper/312:09:203
1486326325,20sleep20-21swapper/209:24:202
1478226321,26sleep114897-21sed07:39:311
144626320,21sleep20-21swapper/210:24:252
821626214,33sleep28258-21sort07:19:252
6492625,48sleep20-21swapper/210:21:012
2750826221,26sleep10-21swapper/110:04:211
1561926217,21sleep20-21swapper/211:10:112
1561926217,21sleep20-21swapper/211:10:112
92712618,45sleep20-21swapper/212:34:032
3021126118,25sleep230218-21modprobe10:13:592
978726016,17sleep30-21swapper/310:49:333
815826012,27sleep18215-21irqstats07:19:241
702226017,17sleep20-21swapper/208:59:322
2507526020,16sleep30-21swapper/309:54:583
2488626010,31sleep10-21swapper/109:54:561
243926015,29sleep10-21swapper/110:29:001
2352226023,21sleep10-21swapper/108:09:191
181526018,21sleep10-21swapper/112:09:221
1806326016,23sleep10-21swapper/109:34:211
1806326016,23sleep10-21swapper/109:34:211
1330726017,23sleep20-21swapper/209:19:192
1113826019,20sleep10-21swapper/110:54:311
1063926017,19sleep30-21swapper/312:39:013
344925919,25sleep23582-21grep07:04:312
339525915,25sleep10-21swapper/108:49:011
334125914,27sleep30-21swapper/308:48:593
1663025911,31sleep316660-21munin-run11:13:593
1663025911,31sleep316660-21munin-run11:13:593
11372596,12sleep20-21swapper/212:07:262
1060825919,22sleep110699-21mailstats09:09:291
89822584,12sleep30-21swapper/312:33:323
762225815,24sleep10-21swapper/110:44:221
674525817,27sleep16823-21mailstats12:24:291
428425815,24sleep10-21swapper/108:49:341
2449625811,33sleep324497-21sshd08:11:323
2186225815,19sleep30-21swapper/309:44:323
2004525816,24sleep320040-21apt-get11:24:133
1003525821,17sleep10-21swapper/112:34:311
2964425715,21sleep10-21swapper/108:29:141
2712025716,19sleep20-21swapper/208:19:302
2034125716,25sleep320334-21apt-get07:59:163
2034125716,25sleep320334-21apt-get07:59:163
1844825712,28sleep218530-21/usr/sbin/munin09:34:302
1844825712,28sleep218530-21/usr/sbin/munin09:34:302
398525614,27sleep23999-21vmstat12:14:322
3096025615,26sleep230966-21ntp_states10:14:272
308825617,22sleep10-21swapper/112:14:151
2964125615,24sleep329637-21apt-get08:29:123
2525125617,18sleep10-21swapper/111:39:301
2470825619,22sleep324740-21grep11:39:163
1722525614,25sleep117292-21memory11:14:261
1639925623,15sleep10-21swapper/109:29:201
1334425614,26sleep113354-21munin-run11:04:001
1010125614,24sleep210097-21apt-get09:09:142
632525515,22sleep30-21swapper/308:57:313
632525515,22sleep30-21swapper/308:57:313
480125513,18sleep20-21swapper/212:19:212
440825514,24sleep34411-21apt-get07:09:123
326825512,19sleep30-21swapper/310:29:303
2085225514,21sleep10-21swapper/107:59:301
2085225514,21sleep10-21swapper/107:59:301
892825413,24sleep18922-21apt-get10:49:141
374925420,16sleep10-21swapper/107:04:341
3155825414,17sleep30-21swapper/311:59:313
2788825417,20sleep20-21swapper/210:04:302
1984225416,15sleep10-21swapper/109:39:251
1984225416,15sleep10-21swapper/109:39:251
168022546,12sleep20-21swapper/211:14:032
96925315,18sleep30-21swapper/308:39:263
96925315,18sleep30-21swapper/308:39:263
622125312,23sleep26215-21apt-get12:24:142
2990925319,15sleep20-21swapper/211:54:292
2990925319,15sleep20-21swapper/211:54:292
2628725315,21sleep226319-21ntp_states09:59:262
948425213,24sleep29480-21apt-get07:24:132
2590225213,24sleep125937-21df09:59:161
2270325213,15sleep20-21swapper/208:04:362
2189325214,25sleep121894-21http08:04:191
2159925216,20sleep2287-21dbus-daemon08:03:592
2159925216,20sleep2287-21dbus-daemon08:03:592
2004825213,24sleep120116-21df11:24:151
1740825216,18sleep10-21swapper/107:49:201
1343525217,16sleep10-21swapper/109:19:221
1224825213,25sleep212298-21if_eth010:59:212
1211825213,22sleep312124-21diskstats10:59:183
2648025117,20sleep326605-21cat09:59:313
2143425113,17sleep20-21swapper/209:44:272
593025017,21sleep35943-21apt12:23:583
2936725019,15sleep30-21swapper/308:29:003
1890325016,14sleep20-21swapper/211:19:282
1636325016,15sleep20-21swapper/207:44:332
68732496,34sleep20-21swapper/210:41:042
3139424719,15sleep10-21swapper/111:59:291
166072465,30sleep10-21swapper/107:44:441
112412377,13sleep20-21swapper/207:29:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional