You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 07:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Jul 26, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
222642384659,3771sleep021761-21apt-get20:18:140
26374222715,2254sleep315366-1kworker/3:0H22:13:403
31774277016,736sleep031791-21/usr/sbin/munin00:13:150
26807251314,478sleep226880-21systemd-run23:58:032
17163244212,413sleep017211-21ntpq21:43:290
21738238229,339sleep121817-21apt-config20:18:031
1161623280,12sleep10-21swapper/121:28:031
25945230811,278sleep126093-21gpgv22:13:051
243182299114,91sleep10-21swapper/122:08:041
2579022847,242sleep125826-21ntpq20:28:331
2579022847,242sleep125826-21ntpq20:28:331
24562270109,120sleep10-21swapper/100:23:061
24562270109,120sleep10-21swapper/100:23:061
8462225984,156sleep18503-21munin-run21:18:021
12270225715,226sleep212311-21ntpq21:28:292
16046224813,211sleep116080-21ps19:58:331
9547224227,21sleep10-21swapper/119:38:431
9479223745,173sleep19631-21unixbench_multi21:18:391
27408222742,88sleep10-21swapper/120:33:341
27408222742,88sleep10-21swapper/120:33:341
4351222648,130sleep131377-21cyclictest00:28:231
31672222112,193sleep031709-21ntpq22:28:300
31672222112,193sleep031709-21ntpq22:28:300
26114222185,24sleep10-21swapper/123:53:341
3150422180,12sleep131377-21cyclictest19:08:051
6378221754,47sleep10-21swapper/119:28:331
17581221647,57sleep10-21swapper/123:28:221
1457221119,29sleep10-21swapper/120:53:361
3133022057,187sleep131373-21unin-run20:48:031
13786220424,27sleep113393-21/usr/sbin/munin21:33:291
1088220441,84sleep10-21swapper/119:13:191
14096219625,155sleep113982-21/usr/sbin/munin19:53:241
2872219412,60sleep10-21swapper/119:18:241
2872219412,60sleep10-21swapper/119:18:241
20180219412,167sleep120192-21sh20:13:021
15319217513,148sleep315339-21grep21:38:233
3246021720,29sleep232464-21unixbench_singl19:08:352
21905216718,133sleep321756-21latency_hist20:18:053
2785021669,141sleep127990-21sed22:18:061
85521628,115sleep1936-21cut22:33:321
3068421628,118sleep1364-21snmpd00:08:281
7192216040,73sleep17344-21latency_hist00:38:041
16861215811,41sleep10-21swapper/121:43:221
3331215745,64sleep10-21swapper/100:23:391
3331215745,64sleep10-21swapper/100:23:391
30805215119,107sleep330946-21unixbench_singl20:43:373
2359321516,30sleep10-21swapper/120:23:041
1581221519,106sleep118918-1kworker/1:1H23:23:061
28787214916,121sleep128874-21timerandwakeup22:18:421
320821470,17sleep3294-21in:imuxsock22:43:023
9719214045,21sleep10-21swapper/123:03:191
2350213920,100sleep02372-21sendmail19:18:030
21821213916,109sleep121857-21ntp_kernel_err21:58:281
3220221370,9sleep20-21swapper/200:13:292
2367213620,97sleep02454-21readlink00:23:050
2367213620,97sleep02454-21readlink00:23:050
1439521347,37sleep10-21swapper/123:18:171
25151213332,68sleep125204-21latency_hist20:28:041
547321329,81sleep10-21swapper/122:48:301
24558213119,86sleep10-21swapper/123:48:331
3231421306,34sleep10-21swapper/122:30:171
3231421306,34sleep10-21swapper/122:30:171
755021298,83sleep17543-21apt-get19:33:181
755021298,83sleep17543-21apt-get19:33:181
104221290,26sleep212-21rcu_sched00:18:212
104221290,26sleep212-21rcu_sched00:18:212
12574212829,78sleep014708-1kworker/0:1H23:13:030
12574212829,78sleep014708-1kworker/0:1H23:13:030
472721278,83sleep14820-21mailstats19:23:331
472721278,83sleep14820-21mailstats19:23:331
16899212717,94sleep216985-21grep20:03:062
20640212614,98sleep220692-21sort20:13:242
920212319,89sleep2960-21if_eth020:53:252
2173321230,38sleep321737-21unixbench_singl23:38:373
28405211922,81sleep028408-21cron00:03:020
1950421198,75sleep119502-21ntp_states23:33:311
1814821195,10sleep10-21swapper/120:04:551
3267121188,78sleep10-21swapper/120:50:251
29718211827,41sleep10-21swapper/122:23:191
699021168,39sleep10-21swapper/121:13:061
11237211520,70sleep20-21swapper/223:08:132
11237211520,70sleep20-21swapper/223:08:132
32313211476,20sleep112-21rcu_sched19:08:351
18925211321,51sleep20-21swapper/220:08:192
3556211224,68sleep33576-21awk21:03:033
3556211224,68sleep33576-21awk21:03:033
23661211216,78sleep2225-21systemd-journal23:48:022
32531211121,70sleep332540-21modprobe22:33:023
32531211121,70sleep332540-21modprobe22:33:023
28414211025,72sleep11-21systemd00:03:031
4682210910,22sleep10-21swapper/121:03:491
22623210976,18sleep122672-21irqstats23:43:241
3408210719,65sleep23562-21grep00:23:402
3408210719,65sleep23562-21grep00:23:402
28699210710,12sleep30-21swapper/300:03:213
324621066,67sleep10-21swapper/120:59:051
324621066,67sleep10-21swapper/120:59:051
28257210620,65sleep328141-21/usr/sbin/munin22:18:243
18075210614,75sleep018168-21/usr/sbin/munin23:28:340
11927210670,24sleep111956-21munin-run19:48:031
22168210429,33sleep10-21swapper/123:43:041
15261210425,48sleep315280-21idleruntime-cro19:58:033
2429210126,56sleep32555-21wc19:18:063
14396210014,73sleep214431-21grep23:18:182
325429911,72sleep03317-21latency_hist22:43:040
192872996,12sleep10-21swapper/121:49:111
1654629916,42sleep10-21swapper/121:43:061
1244929942,45sleep112446-21apt-get19:48:161
115232997,39sleep10-21swapper/123:08:241
115232997,39sleep10-21swapper/123:08:241
2639329816,60sleep30-21swapper/323:53:363
2631429712,72sleep225830-21apt-get22:13:142
540629513,59sleep35015-21/usr/sbin/munin22:48:283
2585829513,65sleep025859-21gpgv22:13:040
237629421,29sleep00-21swapper/020:58:230
237629421,29sleep00-21swapper/020:58:230
1064729423,31sleep20-21swapper/219:43:162
857529320,33sleep10-21swapper/122:58:271
807629314,57sleep07525-21/usr/sbin/munin19:34:100
807629314,57sleep07525-21/usr/sbin/munin19:34:100
77862937,75sleep10-21swapper/122:57:181
279392935,77sleep00-21swapper/020:34:460
279392935,77sleep00-21swapper/020:34:460
703429217,61sleep2225-21systemd-journal00:37:332
511729216,57sleep35170-21users00:28:373
2300929211,61sleep30-21swapper/320:18:383
1194929110,15sleep30-21swapper/321:28:223
719729020,24sleep00-21swapper/021:13:210
53612905,16sleep00-21swapper/019:26:350
53612905,16sleep00-21swapper/019:26:350
2464629024,32sleep20-21swapper/222:08:222
2403829014,59sleep224096-21cut23:48:222
200232907,13sleep10-21swapper/121:53:231
842928920,46sleep28492-21latency22:58:242
876128813,30sleep20-21swapper/221:18:162
372728820,32sleep10-21swapper/122:43:251
372728820,32sleep10-21swapper/122:43:251
289622889,40sleep10-21swapper/120:38:301
774828711,55sleep27525-21/usr/sbin/munin19:33:352
774828711,55sleep27525-21/usr/sbin/munin19:33:352
52728727,34sleep00-21swapper/019:13:060
2887528724,39sleep229026-21grep22:18:432
1264028720,29sleep10-21swapper/123:13:041
1264028720,29sleep10-21swapper/123:13:041
1233528710,35sleep10-21swapper/121:28:321
2999728625,46sleep130023-21latency_hist00:08:031
1725328612,61sleep317306-21latency_hist23:28:043
1351328618,48sleep013545-21fschecks_count21:33:240
65892857,11sleep00-21swapper/021:09:010
5512855,30sleep10-21swapper/100:17:201
2548228513,51sleep325487-21/usr/sbin/munin20:28:273
2548228513,51sleep325487-21/usr/sbin/munin20:28:273
2023328427,27sleep00-21swapper/020:13:060
1467728422,39sleep314700-21apt-get21:38:033
46828212,55sleep3563-21latency_hist20:53:053
2452928212,53sleep324657-21/usr/sbin/munin20:23:383
792328125,33sleep30-21swapper/322:58:053
489828120,40sleep35030-21tail19:23:353
489828120,40sleep35030-21tail19:23:353
257452817,23sleep00-21swapper/023:53:250
1493128114,49sleep215129-21modprobe21:38:062
31002805,64sleep10-21swapper/122:41:441
268828016,44sleep02203-21apt-get00:23:130
268828016,44sleep02203-21apt-get00:23:130
2819327912,51sleep21-21systemd20:38:032
2819327912,51sleep21-21systemd20:38:032
1242527921,37sleep311942-21apt-get19:48:143
422727823,30sleep30-21swapper/321:03:393
1494327813,45sleep015010-21/usr/sbin/munin23:18:330
911627714,44sleep39135-21apt-get19:38:293
405427720,37sleep23569-21apt-get21:03:142
969227613,49sleep09711-21apt23:03:140
327227612,48sleep33368-21sed19:18:343
327227612,48sleep33368-21sed19:18:343
2191327624,30sleep00-21swapper/021:58:310
1678527514,26sleep30-21swapper/321:43:203
1204527516,43sleep311238-21/usr/sbin/munin23:08:343
1204527516,43sleep311238-21/usr/sbin/munin23:08:343
113927519,23sleep00-21swapper/022:33:350
1010627520,24sleep30-21swapper/321:23:053
2953227420,26sleep00-21swapper/022:23:050
2054327415,39sleep320741-21store23:38:053
2479327317,41sleep324854-21users23:48:353
581627217,34sleep05809-21apt-get00:33:190
468327227,22sleep00-21swapper/000:28:300
208582728,26sleep00-21swapper/023:38:190
2074027217,26sleep00-21swapper/020:13:260
225027123,28sleep32353-21systemd-run00:23:043
225027123,28sleep32353-21systemd-run00:23:043
139282705,11sleep30-21swapper/323:16:053
139282705,11sleep30-21swapper/323:16:053
1025227027,25sleep20-21swapper/223:03:332
3132226917,38sleep031323-21cron20:48:020
1843826920,27sleep318528-21apt-config20:08:043
104112698,16sleep00-21swapper/021:23:220
2851426822,27sleep20-21swapper/220:38:192
2159326812,17sleep20-21swapper/223:38:372
690526720,31sleep26926-21munin-run21:13:032
635026713,40sleep06419-21latency_hist22:53:050
37952676,12sleep00-21swapper/019:21:220
37952676,12sleep00-21swapper/019:21:220
3131626620,27sleep331334-21/usr/sbin/munin19:08:043
2872326612,22sleep00-21swapper/020:38:250
2722826613,15sleep20-21swapper/223:58:232
2393226620,17sleep20-21swapper/220:23:252
132702667,13sleep00-21swapper/019:48:590
480026513,16sleep00-21swapper/022:48:040
480026513,16sleep00-21swapper/022:48:040
1431726514,36sleep214326-21/usr/sbin/munin21:33:382
854026418,26sleep30-21swapper/321:18:043
375026413,34sleep23954-21date21:03:062
375026413,34sleep23954-21date21:03:062
3240326416,24sleep30-21swapper/300:13:353
2249026420,23sleep30-21swapper/323:43:223
905526318,28sleep09103-21sed21:18:300
79812634,17sleep30-21swapper/319:33:403
79812634,17sleep30-21swapper/319:33:403
394526320,21sleep20-21swapper/222:43:302
394526320,21sleep20-21swapper/222:43:302
705526213,27sleep20-21swapper/222:53:312
650526220,19sleep20-21swapper/219:28:342
608026219,22sleep30-21swapper/300:33:263
402826220,20sleep30-21swapper/300:28:053
402826220,20sleep30-21swapper/300:28:053
307472629,39sleep030839-21mailstats00:08:310
2888126215,21sleep00-21swapper/000:03:260
2633826216,25sleep00-21swapper/022:13:160
667226115,21sleep00-21swapper/022:53:210
320226119,26sleep23207-21modprobe22:43:022
3032526111,36sleep230353-21cut00:08:212
2354526113,26sleep30-21swapper/322:03:343
951226014,15sleep00-21swapper/019:38:420
83542606,10sleep20-21swapper/221:16:382
2786426019,25sleep328005-21unixbench_singl23:58:363
2328126012,26sleep023370-21/usr/sbin/munin22:03:270
2205826021,20sleep30-21swapper/321:58:343
1950526010,36sleep219523-21open_inodes23:33:322
1534226017,27sleep00-21swapper/021:38:240
1321026017,24sleep20-21swapper/223:13:302
1321026017,24sleep20-21swapper/223:13:302
450425913,29sleep04537-21/usr/sbin/munin21:03:440
357925919,21sleep30-21swapper/322:43:233
357925919,21sleep30-21swapper/322:43:233
3011825915,25sleep30-21swapper/322:23:293
3009125916,21sleep00-21swapper/019:03:220
2277325913,25sleep00-21swapper/023:43:270
1615725914,31sleep316235-21ls19:58:353
1032325922,19sleep30-21swapper/323:03:343
96772580,6sleep00-21swapper/023:03:050
629625812,29sleep36422-21sed21:08:403
1281525814,27sleep012808-21apt-get23:13:200
1281525814,27sleep012808-21apt-get23:13:200
941225713,20sleep20-21swapper/219:38:402
71325715,25sleep0707-21apt-get20:53:190
67602576,39sleep00-21swapper/019:29:410
3162225715,25sleep231689-21df20:48:202
3016825716,26sleep030170-21ntp_states22:23:300
2595925717,24sleep20-21swapper/220:28:382
2595925717,24sleep20-21swapper/220:28:382
2279325720,17sleep20-21swapper/222:03:052
2029925716,26sleep020363-21http23:38:020
2014525714,26sleep320151-21grep21:53:253
186942575,10sleep30-21swapper/323:32:203
68725613,26sleep2716-21ntp_states19:13:082
397825619,19sleep20-21swapper/219:23:052
397825619,19sleep20-21swapper/219:23:052
3000025615,24sleep229996-21apt-get19:03:162
2851025615,25sleep328507-21apt-get20:38:163
2113525612,27sleep021144-21modprobe21:58:020
1795525616,26sleep217992-21ps23:28:322
1736225615,20sleep20-21swapper/221:43:352
1688825620,17sleep00-21swapper/020:03:040
1376925618,19sleep20-21swapper/219:53:052
448525516,18sleep20-21swapper/219:23:262
448525516,18sleep20-21swapper/219:23:262
2826525513,28sleep328298-21latency_hist20:38:043
2826525513,28sleep328298-21latency_hist20:38:043
1731925514,19sleep00-21swapper/020:03:250
1084325514,24sleep210927-21mailstats21:23:332
321922547,13sleep00-21swapper/020:48:340
3213525417,18sleep30-21swapper/319:08:333
2581425415,25sleep325842-21munin-run22:13:033
2416325414,21sleep024220-21irqstats23:48:240
193752547,38sleep00-21swapper/021:50:190
1841725412,21sleep30-21swapper/321:48:243
1740525418,18sleep20-21swapper/220:03:272
16892545,14sleep30-21swapper/320:54:273
1195725412,25sleep011989-21tune2fs21:28:230
104992545,12sleep00-21swapper/019:43:060
435525315,21sleep20-21swapper/200:28:242
323972536,15sleep20-21swapper/222:31:192
323972536,15sleep20-21swapper/222:31:192
3005625312,22sleep230054-21apt-get20:43:152
3002425311,17sleep20-21swapper/200:08:042
2431625310,21sleep00-21swapper/020:23:350
1963025313,24sleep219639-21/usr/sbin/munin21:53:042
314812525,38sleep30-21swapper/300:12:423
1888425214,22sleep318423-21apt-get20:08:083
1618925111,23sleep216212-21irqstats23:23:232
1417625112,20sleep20-21swapper/219:53:252
953825015,15sleep30-21swapper/323:03:053
79625014,21sleep2820-21ntp_states22:33:292
2232425012,22sleep222367-21df20:18:202
1445925012,15sleep00-21swapper/019:53:320
1071825015,18sleep30-21swapper/319:43:213
3102524917,16sleep20-21swapper/222:28:042
2302924913,22sleep223082-21df22:03:212
156292486,31sleep30-21swapper/323:22:433
706424713,18sleep30-21swapper/319:33:033
633724711,23sleep26344-21systemctl22:53:032
2027524714,19sleep20-21swapper/221:53:282
1032424711,21sleep310319-21apt-get21:23:173
1594424615,15sleep00-21swapper/023:23:170
35624510,17sleep30-21swapper/322:33:183
2537424512,18sleep00-21swapper/020:28:160
2537424512,18sleep00-21swapper/020:28:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional