You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 06:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Sat May 09, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2194221561922,15580sleep38271-21kworker/3:100:19:593
27930237694,105sleep228120-21store19:34:492
23850237213,256sleep223948-21fschecks_time22:45:042
829236346,299sleep2866-21ntpq19:50:112
829236346,299sleep2866-21ntpq19:50:112
990623400,89sleep219574-21cyclictest20:20:052
19176225863,183sleep218771-21/usr/sbin/munin00:10:092
11043225279,155sleep211118-21latency_hist22:04:472
6263225020,215sleep36286-21pluginstate23:30:143
20726224318,210sleep120833-21tune2fs22:35:051
262322110,99sleep20-21swapper/223:20:032
262322110,99sleep20-21swapper/223:20:032
28944220715,168sleep129077-21grep19:36:501
12836219010,89sleep212808-21/usr/sbin/munin22:09:592
2414218725,74sleep22567-21unixbench_singl21:35:152
2414218725,74sleep22567-21unixbench_singl21:35:152
9564218419,149sleep39618-21latency_hist20:19:473
5535218024,68sleep25599-21sendmail_mailst21:45:132
12614217641,59sleep20-21swapper/222:09:472
16956217221,132sleep216976-21idleruntime-cro00:04:442
16956217221,132sleep216976-21idleruntime-cro00:04:442
18876216420,125sleep318771-21/usr/sbin/munin00:10:023
2290216312,136sleep12413-21cat21:35:141
2290216312,136sleep12413-21cat21:35:141
8416216028,111sleep28480-21latency21:55:062
12663215811,134sleep212706-21latency_hist20:29:472
18093215511,124sleep20-21swapper/220:45:212
18093215511,124sleep20-21swapper/220:45:212
11790215416,119sleep111884-21grep22:05:141
7238215317,21sleep00-21swapper/020:10:150
597521530,26sleep05979-21unixbench_singl20:05:180
385215349,87sleep20-21swapper/221:30:072
24792215310,88sleep224815-21sort00:29:462
32335215110,87sleep232345-21unin-run21:29:462
26785215111,126sleep326391-21/usr/sbin/munin19:30:103
20386215115,73sleep220445-21latency_hist22:34:462
29939214863,49sleep21-21systemd23:04:462
1839214813,119sleep11840-21sshd23:15:591
27093214678,18sleep20-21swapper/200:35:152
27093214678,18sleep20-21swapper/200:35:152
20357214177,22sleep20-21swapper/200:15:012
25175213534,86sleep225309-21cat21:05:162
29136213424,93sleep329171-21grep23:00:143
16038213428,88sleep10-21swapper/120:39:471
7717213217,96sleep00-21swapper/023:35:070
7905212818,96sleep37949-21date21:54:463
30504212610,97sleep21-21systemd21:24:462
2736421258,97sleep00-21swapper/022:55:070
1555212522,83sleep21699-21grep23:15:152
26479212417,89sleep20-21swapper/221:10:082
15358212426,77sleep215493-21apt-config23:59:482
6375212312,94sleep16447-21switchtime23:30:161
25312212136,66sleep219574-21cyclictest22:49:482
24306211912,51sleep20-21swapper/221:04:472
12048211913,84sleep312049-21munin-node23:49:563
9780211715,80sleep19790-21acpi20:19:571
24028211723,68sleep324186-21unixbench_singl19:20:173
24028211723,68sleep324186-21unixbench_singl19:20:173
12525211730,18sleep20-21swapper/223:50:112
761921165,10sleep20-21swapper/221:51:432
31540211411,85sleep331664-21latency_hist23:09:473
2560821149,81sleep30-21swapper/319:25:173
1681211312,83sleep31751-21df21:35:003
1681211312,83sleep31751-21df21:35:003
4507211234,63sleep20-21swapper/220:00:452
2621321115,11sleep20-21swapper/200:33:382
2621321115,11sleep20-21swapper/200:33:382
6457211016,75sleep16617-21latency_hist20:09:481
22672211012,81sleep022674-21cron20:59:450
22453210940,56sleep20-21swapper/200:20:142
7433210823,65sleep37451-21fschecks_count23:35:023
28645210815,76sleep128646-21munin-node22:59:561
13308210816,74sleep113338-21/usr/sbin/munin20:30:101
4812210621,29sleep10-21swapper/121:44:471
19344210614,77sleep019422-21sed22:30:080
32678210516,72sleep30-21swapper/319:49:483
28303210514,31sleep10-21swapper/121:15:161
1390210425,60sleep01414-21sh21:34:450
10380210414,72sleep010456-21switchtime20:20:160
16738210331,16sleep216772-21ps20:40:132
631210218,17sleep20-21swapper/223:14:472
17606210222,60sleep017670-21sort22:25:030
17606210222,60sleep017670-21sort22:25:030
3113021018,37sleep2363-21snmpd23:05:182
303821016,12sleep20-21swapper/219:57:092
23440210111,75sleep023528-21/usr/sbin/munin19:20:050
23440210111,75sleep023528-21/usr/sbin/munin19:20:050
20695210121,13sleep20-21swapper/222:35:022
10258210127,17sleep210285-21munin-run23:44:462
8211210025,16sleep20-21swapper/220:14:582
25310210018,66sleep325423-21grep21:05:173
21417210025,17sleep20-21swapper/220:55:022
3264129924,58sleep20-21swapper/219:49:462
3264129924,58sleep20-21swapper/219:49:462
509129724,15sleep20-21swapper/220:04:562
527029519,34sleep25287-21modprobe23:29:442
2724129510,22sleep227309-21basename22:55:052
2540729517,61sleep125416-21/usr/sbin/munin00:30:101
2540729517,61sleep125416-21/usr/sbin/munin00:30:101
146492940,6sleep00-21swapper/020:34:490
2636229320,51sleep326515-21latency_hist00:34:483
2636229320,51sleep326515-21latency_hist00:34:483
1858729336,33sleep118632-21awk19:05:091
1752229311,69sleep117590-21sh22:25:001
1752229311,69sleep117590-21sh22:25:001
161542936,75sleep216214-21cat22:20:052
161542936,75sleep216214-21cat22:20:052
84042925,76sleep20-21swapper/223:37:072
1947229218,49sleep019604-21timerwakeupswit00:10:150
399929015,57sleep24061-21find21:40:322
2454429024,35sleep00-21swapper/021:04:580
221952908,20sleep222197-21apt-get22:39:552
221952908,20sleep222197-21apt-get22:39:552
1387629015,58sleep313851-21apt-get23:55:083
3234828918,54sleep032495-21latency_hist21:29:480
296172880,6sleep00-21swapper/021:20:070
140502885,72sleep20-21swapper/220:33:142
310462878,66sleep231069-21munin-run19:44:462
2900828713,59sleep029126-21date21:19:470
283202877,20sleep20-21swapper/219:35:252
2275228724,37sleep30-21swapper/320:59:483
711128616,42sleep00-21swapper/023:34:470
2032828620,47sleep120336-21acpi00:14:561
181982865,70sleep20-21swapper/200:05:492
289022855,68sleep20-21swapper/221:19:252
2102228517,51sleep321213-21dpkg20:54:473
373528414,54sleep33792-21irqstats21:40:253
270822845,67sleep00-21swapper/021:11:030
2392528449,18sleep20-21swapper/219:20:162
2392528449,18sleep20-21swapper/219:20:162
350228311,57sleep03482-21apt-get21:40:020
102932830,6sleep30-21swapper/322:00:153
2514828219,40sleep125282-21gpgv22:49:471
244492820,7sleep30-21swapper/321:04:473
1511328249,18sleep20-21swapper/222:15:162
469128115,50sleep14827-21awk23:25:151
3214428117,46sleep332171-21awk23:10:083
1196528121,37sleep312113-21grep22:05:163
2709528017,18sleep10-21swapper/122:55:001
2648928017,34sleep00-21swapper/019:30:020
254528019,31sleep10-21swapper/123:20:031
254528019,31sleep10-21swapper/123:20:031
198128013,48sleep02049-21/usr/sbin/munin19:55:010
1855928013,50sleep018528-21latency_hist00:09:470
1573728011,54sleep315738-21sshd20:36:593
1493028012,50sleep014998-21/usr/sbin/munin22:15:140
860727916,45sleep38622-21open_inodes21:55:113
3217327917,47sleep032210-21sed23:10:090
1626727914,49sleep116324-21switchtime00:00:271
1626727914,49sleep116324-21switchtime00:00:271
1480427910,55sleep20-21swapper/220:35:202
520627824,33sleep30-21swapper/320:05:063
1843927814,18sleep20-21swapper/219:05:062
487927719,31sleep10-21swapper/120:04:471
3057327714,42sleep330683-21latency_hist21:24:473
625627643,15sleep20-21swapper/223:30:132
3142427616,40sleep131514-21fschecks_time19:45:051
3142427616,40sleep131514-21fschecks_time19:45:051
2641527623,29sleep10-21swapper/119:29:591
228432765,50sleep219574-21cyclictest19:17:452
1727227618,43sleep017291-21latency_hist22:24:470
1727227618,43sleep017291-21latency_hist22:24:470
91882757,55sleep00-21swapper/020:16:050
2357627528,24sleep30-21swapper/300:25:033
200172755,12sleep20-21swapper/220:50:102
581327411,46sleep15932-21awk21:45:161
430127416,34sleep30-21swapper/320:00:173
260772747,11sleep20-21swapper/219:29:082
1883227416,42sleep318931-21latency_hist22:29:473
1883227416,42sleep318931-21latency_hist22:29:473
1695327422,25sleep10-21swapper/120:40:161
996527319,34sleep39962-21apt-get23:41:403
445927315,24sleep00-21swapper/020:00:180
2630027319,35sleep026303-21cron00:34:450
2630027319,35sleep026303-21cron00:34:450
1387827320,30sleep00-21swapper/023:55:100
76872725,11sleep20-21swapper/220:11:432
3114527211,47sleep131199-21grep21:25:211
2784727221,34sleep327856-21modprobe00:39:443
2784727221,34sleep327856-21modprobe00:39:443
208222729,50sleep120825-21sshd19:11:591
1962727213,43sleep119651-21cpu20:50:011
184822725,11sleep20-21swapper/222:25:582
184822725,11sleep20-21swapper/222:25:582
1183627212,21sleep211947-21latency_hist23:49:472
207812715,12sleep20-21swapper/219:11:272
1292127119,24sleep10-21swapper/122:10:021
715327015,39sleep37234-21grep20:10:143
3032727017,36sleep130417-21grep23:05:051
281992708,29sleep30-21swapper/319:34:583
1906927021,31sleep30-21swapper/322:29:593
1865327023,31sleep018692-21wc19:05:120
1344127017,26sleep00-21swapper/022:10:140
1305527013,26sleep00-21swapper/020:30:050
1197227022,19sleep00-21swapper/020:25:170
1029227019,27sleep30-21swapper/320:20:153
63226920,25sleep10-21swapper/121:30:141
3020426912,44sleep030208-21find23:04:560
98632687,50sleep00-21swapper/023:40:360
868126818,34sleep18751-21/usr/sbin/munin21:55:131
2507526811,43sleep025125-21if_eth019:25:060
210062687,14sleep00-21swapper/019:14:200
1806026817,35sleep118051-21apt-get20:45:001
1806026817,35sleep118051-21apt-get20:45:001
1619526814,41sleep016266-21tr00:00:260
1619526814,41sleep016266-21tr00:00:260
1254526816,34sleep012605-21/usr/sbin/munin23:50:130
2619726712,34sleep126284-21fschecks_time21:10:031
2262826723,24sleep00-21swapper/000:20:160
2231726714,36sleep122344-21/usr/sbin/munin00:20:091
2401326624,25sleep00-21swapper/022:45:060
1109726616,26sleep00-21swapper/023:45:170
80826519,22sleep30-21swapper/321:30:163
795426518,21sleep00-21swapper/021:54:470
675526517,26sleep10-21swapper/120:10:041
297562656,11sleep10-21swapper/119:39:521
266512655,50sleep00-21swapper/022:52:450
2235626516,29sleep00-21swapper/022:40:050
2235626516,29sleep00-21swapper/022:40:050
2214726514,34sleep122184-21ntp_states19:15:101
52626424,24sleep10-21swapper/119:50:061
52626424,24sleep10-21swapper/119:50:061
1887126424,22sleep30-21swapper/319:05:153
3045126318,23sleep00-21swapper/019:40:160
2120026320,27sleep321273-21/usr/sbin/munin22:35:163
1465626314,22sleep10-21swapper/122:15:071
1228526312,14sleep10-21swapper/123:50:061
1091926321,28sleep310947-21sed23:45:113
1023626314,31sleep110256-21latency_hist23:44:451
636126216,23sleep00-21swapper/021:49:470
535026222,21sleep00-21swapper/023:29:460
254132627,40sleep10-21swapper/119:25:151
2336026215,25sleep10-21swapper/121:00:101
1795726218,26sleep118102-21/usr/sbin/munin00:05:161
323826117,29sleep33248-21modprobe19:59:453
2854126111,14sleep00-21swapper/019:36:400
18992615,12sleep30-21swapper/323:16:463
1753326116,28sleep317572-21http20:44:463
1270726111,34sleep312860-21cat20:29:483
1080026112,18sleep10-21swapper/123:45:091
633426017,28sleep36359-21munin-run21:49:463
3160726018,20sleep00-21swapper/019:45:080
3160726018,20sleep00-21swapper/019:45:080
2917426017,21sleep00-21swapper/023:00:160
2575926013,29sleep125862-21perl22:51:281
2462126021,19sleep10-21swapper/119:24:471
2462126021,19sleep10-21swapper/119:24:471
1597026023,18sleep10-21swapper/122:20:001
1597026023,18sleep10-21swapper/122:20:001
3221125911,33sleep132230-21ntp_states23:10:101
2386125917,28sleep00-21swapper/000:25:080
2033725918,25sleep120385-21systemd-run22:34:451
529125813,27sleep3284-21dbus-daemon23:29:453
371725814,28sleep33734-21awk23:24:453
371725814,28sleep33734-21awk23:24:453
2496525817,23sleep125001-21ntp_states21:05:111
2170625819,19sleep30-21swapper/300:19:463
1145225815,18sleep10-21swapper/120:25:061
973025716,26sleep09719-21apt-get22:00:000
2677525714,26sleep126838-21/usr/sbin/munin00:35:061
2677525714,26sleep126838-21/usr/sbin/munin00:35:061
2043425711,33sleep020329-21/usr/sbin/munin00:15:030
2395025617,18sleep30-21swapper/322:45:053
2326525613,26sleep023359-21memory21:00:090
2187725613,25sleep121883-21ntp_states20:55:181
1960525615,24sleep019617-21apt-get20:49:560
1805525614,27sleep317550-21apt-get20:44:583
1805525614,27sleep317550-21apt-get20:44:583
1654625614,25sleep016592-21sed20:40:070
232002555,39sleep30-21swapper/322:41:213
232002555,39sleep30-21swapper/322:41:213
911325417,18sleep10-21swapper/120:15:171
660225413,24sleep16670-21df21:50:001
3133325415,22sleep331329-21apt-get19:44:593
2191425412,21sleep00-21swapper/019:15:050
1524125413,24sleep115357-21cat23:59:471
2948225318,18sleep10-21swapper/121:20:041
2899425313,24sleep329005-21munin-run21:19:463
272925314,17sleep00-21swapper/023:20:060
272925314,17sleep00-21swapper/023:20:060
2553825313,23sleep325030-21apt-get22:49:573
2127525317,16sleep00-21swapper/022:35:170
1926025318,17sleep10-21swapper/100:10:121
1805725310,25sleep00-21swapper/020:44:590
1805725310,25sleep00-21swapper/020:44:590
175325310,19sleep00-21swapper/021:35:010
175325310,19sleep00-21swapper/021:35:010
1437925313,24sleep314376-21apt-get22:14:573
3105625213,22sleep331022-21apt-get21:25:153
2737025214,24sleep327390-21/usr/sbin/munin21:14:463
2276125213,19sleep10-21swapper/122:40:141
2276125213,19sleep10-21swapper/122:40:141
1621525213,22sleep316241-21irqstats22:20:063
1621525213,22sleep316241-21irqstats22:20:063
3036125116,16sleep30-21swapper/319:40:153
2659125112,20sleep00-21swapper/000:35:010
2659125112,20sleep00-21swapper/000:35:010
2172925116,16sleep30-21swapper/319:15:003
155002516,15sleep30-21swapper/323:59:493
155002516,15sleep30-21swapper/323:59:493
138025114,16sleep00-21swapper/023:15:130
1356125118,18sleep30-21swapper/320:30:173
1099425116,20sleep10-21swapper/122:04:461
2475125011,21sleep124776-21munin-run00:29:451
1127825013,20sleep011274-21apt-get22:04:570
774924914,14sleep10-21swapper/123:35:081
42924912,23sleep0421-21apt-get19:50:010
42924912,23sleep0421-21apt-get19:50:010
86824810,23sleep3887-21ntp_states19:50:123
86824810,23sleep3887-21ntp_states19:50:123
205024811,22sleep12056-21df_inode19:55:021
1323824813,21sleep313287-21cat22:10:093
271682478,17sleep30-21swapper/322:55:023
177812478,14sleep30-21swapper/300:05:143
202332446,14sleep30-21swapper/319:10:113
69012428,15sleep00-21swapper/021:50:070
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional