You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 01:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Apr 22, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
82702701720,6981sleep14405-21kworker/1:110:49:521
2688223227,2300sleep3546-21kworker/3:112:16:083
22133291913,888sleep222201-21df09:51:082
29010261015,576sleep029086-21/usr/sbin/munin10:11:220
24887246241,397sleep324901-21fschecks_count08:16:143
5018236521,327sleep05076-21/usr/sbin/munin07:11:250
1166123528,332sleep311689-21ntpq09:16:163
23468234620,309sleep323561-21latency_hist09:55:543
7177233617,136sleep30-21swapper/310:46:103
5421232412,112sleep30-21swapper/308:56:173
18561232414,69sleep30-21swapper/307:56:073
314152315115,187sleep331448-21fschecks_time08:36:263
314152315115,187sleep331448-21fschecks_time08:36:263
25544231324,267sleep025693-21grep08:16:280
9914228812,259sleep09976-21unin-run10:55:520
162132284103,161sleep316295-21latency_hist11:15:553
307062283107,55sleep30-21swapper/310:16:383
26568227691,71sleep326572-21modprobe10:05:513
24009227098,154sleep324158-21latency_hist11:40:553
2949222669,163sleep329515-21df_inode08:31:093
32388226311,154sleep332502-21timerandwakeup10:21:223
18173225551,77sleep318322-21unixbench_multi09:36:233
120192243159,34sleep312051-21cron07:35:523
120192243159,34sleep312051-21cron07:35:523
841422207,193sleep313441-21snmpd09:06:143
7882220616,20sleep30-21swapper/307:21:183
4708220368,117sleep34723-21kworker/u8:208:55:523
4708220368,117sleep34723-21kworker/u8:208:55:523
32001220146,74sleep332049-21awk12:06:073
4464218513,148sleep30-21swapper/310:36:213
5149218319,140sleep25257-21timerandwakeup07:11:272
993421769,58sleep30-21swapper/307:26:353
993421769,58sleep30-21swapper/307:26:353
15893217435,126sleep315903-21open_files07:46:183
23215217111,56sleep30-21swapper/308:11:103
23215217111,56sleep30-21swapper/308:11:103
14350217118,133sleep314374-21proc_pri07:41:213
25565216912,139sleep325714-21latency_hist11:45:553
8098216715,127sleep30-21swapper/312:31:193
16458216511,51sleep30-21swapper/309:31:233
7897216110,42sleep30-21swapper/309:05:543
3680215614,124sleep3292-21in:imuxsock10:35:523
6519214532,101sleep36555-21ntpq07:16:213
1976213535,80sleep30-21swapper/308:46:113
28448213328,83sleep30-21swapper/308:26:243
1661213027,87sleep31809-21date08:45:553
16956212935,15sleep30-21swapper/307:51:023
1281421288,59sleep30-21swapper/309:21:053
26143212725,40sleep30-21swapper/311:46:163
13599212723,16sleep30-21swapper/307:40:533
16940212650,39sleep317002-21mailstats11:16:233
16940212650,39sleep317002-21mailstats11:16:233
30453212318,18sleep30-21swapper/312:01:063
2559212311,99sleep32569-21find07:06:033
19497212314,88sleep119521-21/usr/sbin/munin09:41:181
12488212118,79sleep212528-21grep07:36:152
25022211910,50sleep30-21swapper/310:00:553
16821211718,35sleep10-21swapper/111:16:171
16821211718,35sleep10-21swapper/111:16:171
9253211643,20sleep30-21swapper/310:51:233
724821166,86sleep00-21swapper/010:46:110
1527211647,21sleep30-21swapper/310:26:233
9773211518,18sleep30-21swapper/309:11:113
24174211515,79sleep124212-21/usr/sbin/munin09:56:201
2406211534,69sleep32461-21latency_hist07:05:553
27195211422,12sleep30-21swapper/310:06:163
21662211418,55sleep30-21swapper/308:06:053
28403211332,53sleep128397-21apt-get10:11:071
22034211310,87sleep222067-21/usr/sbin/munin08:06:162
16133211320,77sleep016142-21modprobe11:15:520
29732211211,87sleep329797-21cat10:15:533
578021118,59sleep30-21swapper/312:25:543
12565211114,81sleep012578-21munin-run09:20:530
5002211012,39sleep30-21swapper/307:11:243
3021621108,87sleep330293-21latency_hist12:00:543
18965211022,33sleep30-21swapper/311:22:093
27404210915,21sleep10-21swapper/110:06:221
1553210919,11sleep30-21swapper/312:11:203
1358721099,87sleep313655-21ls11:06:133
20580210820,31sleep30-21swapper/309:46:053
7187210711,82sleep07262-21timerandwakeup09:01:230
12981210713,78sleep113071-21timerandwakeup07:36:261
8909210619,67sleep29043-21latency_hist12:35:542
2744721068,18sleep327531-21perl11:51:123
25867210623,60sleep225949-21cat10:01:222
1714521060,10sleep10-21swapper/107:51:121
7390210516,62sleep00-21swapper/012:30:550
25286210413,75sleep125348-21sendmail_mailst08:16:251
20856210410,82sleep320858-21munin-run11:30:533
20671210211,77sleep120732-21fschecks_time09:46:111
3078729916,61sleep230199-21/usr/sbin/munin10:16:412
2967229920,59sleep229731-21basename10:15:522
2733829920,61sleep027402-21grep10:06:210
2242629927,58sleep30-21swapper/311:35:543
1486029912,41sleep30-21swapper/309:26:223
600729814,69sleep16044-21ps10:41:191
82752978,73sleep08350-21xargs07:21:270
416129717,51sleep14362-21apt-config12:20:551
230729723,55sleep12328-21munin-run07:05:531
3095629617,66sleep131034-21run-parts12:01:201
278529614,60sleep02874-21fschecks_time12:16:130
2623029618,56sleep226236-21ntp_states11:46:192
2264829616,65sleep022662-21acpi11:36:030
1397429414,68sleep214000-21fschecks_count07:41:122
3042429131,43sleep130473-21idleruntime10:16:321
2789029118,58sleep127962-21sed11:51:231
685628914,51sleep06984-21ls10:45:550
3034128924,45sleep030423-21cut10:16:310
210572886,27sleep30-21swapper/308:01:493
210572886,27sleep30-21swapper/308:01:493
96328720,43sleep01057-21fschecks_time10:26:120
66292876,28sleep30-21swapper/310:43:403
3250628720,38sleep032653-21head10:21:230
1946428711,35sleep30-21swapper/309:41:163
1216728721,11sleep30-21swapper/311:01:173
1601028622,30sleep10-21swapper/109:31:121
824828525,26sleep10-21swapper/112:31:211
2889828516,45sleep228872-21/usr/sbin/munin11:56:082
210028512,58sleep12141-21if_eth008:46:131
83728412,19sleep10-21swapper/108:41:211
3265528414,57sleep132679-21who10:21:241
2175228413,16sleep10-21swapper/108:06:111
1462928418,50sleep014760-21latency_hist11:10:550
2422128319,42sleep224222-21cpu11:41:062
165128325,36sleep21658-21cat08:45:542
113012835,27sleep30-21swapper/310:58:343
69892826,28sleep30-21swapper/312:27:073
273832825,26sleep30-21swapper/308:23:053
1241328211,53sleep00-21swapper/007:36:090
33228115,23sleep00-21swapper/008:41:070
297122809,16sleep20-21swapper/208:31:162
225828028,34sleep02291-21munin-run07:05:520
2020328027,24sleep20-21swapper/208:01:112
2020328027,24sleep20-21swapper/208:01:112
1087828017,35sleep20-21swapper/210:56:232
2189227920,37sleep022021-21latency_hist09:50:540
1680327818,37sleep216818-21/usr/sbin/munin09:31:262
1257927823,34sleep112674-21latency_hist09:20:541
1154927822,26sleep10-21swapper/111:00:541
561327720,32sleep25623-21df_inode10:41:082
262527710,18sleep00-21swapper/008:46:240
226652777,42sleep10-21swapper/111:36:041
1808627722,36sleep218150-21fschecks_time11:21:112
525527621,32sleep20-21swapper/210:40:522
499827616,45sleep14999-21munin-node08:56:031
1412827618,30sleep114254-21latency_hist09:25:551
2866027521,31sleep00-21swapper/011:55:540
2799027516,42sleep128078-21/usr/sbin/munin08:26:141
144427519,25sleep20-21swapper/210:26:222
1127627520,38sleep011366-21grep07:31:250
1127627520,38sleep011366-21grep07:31:250
2146627417,45sleep221473-21grep11:31:162
1824527419,37sleep018256-21modprobe07:55:520
892427317,40sleep18925-21cron07:25:521
85427327,21sleep20-21swapper/212:10:542
3237127310,15sleep00-21swapper/012:06:160
1596827317,40sleep116038-21sensors_temp07:46:201
75712720,7sleep20-21swapper/207:20:552
3095927216,36sleep131147-21latency_hist08:35:551
2643527120,27sleep10-21swapper/108:21:131
2500927114,27sleep00-21swapper/011:41:240
2297527121,26sleep00-21swapper/008:10:540
1431927115,41sleep114342-21ntp_states07:41:191
423027020,24sleep00-21swapper/008:51:250
423027020,24sleep00-21swapper/008:51:250
305327017,36sleep03095-21wc07:06:250
2807927014,28sleep20-21swapper/208:26:152
2266927011,44sleep122739-21sensors09:51:241
1905727012,29sleep10-21swapper/107:56:221
1127127025,27sleep20-21swapper/209:16:072
527926917,31sleep15373-21grep07:11:351
388526919,31sleep13894-21munin-run07:10:541
3045526912,38sleep030522-21df12:01:070
2713026918,25sleep20-21swapper/210:06:152
2598326920,25sleep00-21swapper/011:46:130
1289126913,24sleep00-21swapper/009:21:100
2015826812,44sleep120255-21timerandwakeup11:26:231
1911926813,35sleep019009-21/usr/sbin/munin09:41:090
1407526817,34sleep214077-21cron09:25:522
1186826815,36sleep211977-21perl11:01:122
997726720,30sleep10-21swapper/110:55:531
893726727,21sleep20-21swapper/207:25:532
1717026722,27sleep017227-21sendmail-msp09:35:520
1445326722,30sleep114509-21tune2fs09:26:121
1047226715,36sleep110517-21systemd-run07:30:531
1047226715,36sleep110517-21systemd-run07:30:531
819226620,27sleep10-21swapper/109:06:081
615426610,42sleep06155-21sshd08:59:090
1763826613,40sleep017703-21idleruntime09:36:120
1463626624,23sleep00-21swapper/009:26:150
1091026516,32sleep210735-21/usr/sbin/munin07:31:142
1091026516,32sleep210735-21/usr/sbin/munin07:31:142
1088426519,21sleep10-21swapper/107:31:131
1088426519,21sleep10-21swapper/107:31:131
2382026418,27sleep10-21swapper/108:11:241
2382026418,27sleep10-21swapper/108:11:241
1807226413,35sleep018080-21sshd11:21:090
1738626413,36sleep217431-21/usr/sbin/munin07:51:182
1556026422,23sleep10-21swapper/111:11:241
1307926416,21sleep00-21swapper/011:05:540
93626313,33sleep1944-21/usr/sbin/munin10:26:091
2244326317,23sleep00-21swapper/011:35:550
127826322,24sleep20-21swapper/207:01:222
1175026311,27sleep10-21swapper/111:01:021
106982637,12sleep00-21swapper/009:12:420
766326218,20sleep20-21swapper/210:46:212
1557126218,28sleep015643-21fw_packets07:46:110
987326113,32sleep19909-21/usr/sbin/munin09:11:131
582026120,20sleep00-21swapper/012:25:550
3016526125,20sleep10-21swapper/112:00:521
288826119,27sleep02912-21/usr/sbin/munin10:31:210
272332610,6sleep20-21swapper/208:21:252
1934826114,20sleep00-21swapper/007:56:250
1103426117,29sleep210974-21latency_hist09:15:552
950026021,20sleep20-21swapper/207:26:262
950026021,20sleep20-21swapper/207:26:262
77092606,11sleep10-21swapper/109:04:141
611826018,25sleep16113-21apt-get07:16:071
2184126016,18sleep20-21swapper/209:50:522
891225912,24sleep00-21swapper/009:06:240
789125918,30sleep27896-21systemd-run09:05:532
629025911,30sleep26331-21systemd-run09:00:522
479225915,24sleep20-21swapper/212:21:362
451025914,20sleep00-21swapper/010:36:220
2142825917,19sleep10-21swapper/108:05:551
2142825917,19sleep10-21swapper/108:05:551
1952525915,26sleep219544-21apt-get11:26:032
1776225918,18sleep20-21swapper/211:20:542
1776225918,18sleep20-21swapper/211:20:542
869625818,23sleep28778-21switchtime09:06:222
732325814,21sleep20-21swapper/212:30:532
667825817,22sleep00-21swapper/007:16:260
663825817,22sleep00-21swapper/012:26:230
324225818,25sleep23314-21sort12:16:232
3221225821,20sleep10-21swapper/112:06:131
3175925810,24sleep20-21swapper/212:05:532
2330125811,26sleep20-21swapper/208:11:132
2330125811,26sleep20-21swapper/208:11:132
2011425816,26sleep020109-21apt-get08:01:050
2011425816,26sleep020109-21apt-get08:01:050
1913725820,18sleep20-21swapper/207:56:232
1457725815,26sleep114602-21munin-run11:10:521
136325825,18sleep11439-21sed12:11:141
674325711,29sleep15997-21/usr/sbin/munin12:26:241
530825714,28sleep15440-21latency_hist10:40:551
444225713,24sleep10-21swapper/112:21:031
3188225722,17sleep20-21swapper/208:36:372
3188225722,17sleep20-21swapper/208:36:372
2662025724,17sleep10-21swapper/110:05:541
62625612,26sleep20-21swapper/210:25:542
2786825612,25sleep027887-21apt-get08:26:050
2400225610,30sleep024075-21memory09:56:150
2397825619,20sleep20-21swapper/211:40:542
1493725612,17sleep20-21swapper/211:11:092
291325514,19sleep10-21swapper/110:31:221
2732425515,24sleep227318-21apt-get11:51:072
2696025519,18sleep00-21swapper/008:21:240
2421325515,16sleep20-21swapper/209:56:212
188925514,24sleep21883-21apt-get08:46:062
1644125515,23sleep016444-21apt-get11:16:040
1644125515,23sleep016444-21apt-get11:16:040
1386125510,23sleep20-21swapper/211:06:212
3122125415,25sleep030715-21apt-get08:36:020
3122125415,25sleep030715-21apt-get08:36:020
2635125414,16sleep10-21swapper/111:46:221
2610725413,19sleep20-21swapper/208:20:542
2525625415,22sleep025252-21apt-get10:01:040
1855425420,17sleep10-21swapper/111:21:221
863725313,24sleep28633-21apt-get10:51:062
2897625312,24sleep028982-21diskstats11:56:100
258425312,23sleep22580-21apt-get07:06:062
2240325318,23sleep022453-21sed09:51:150
344825213,22sleep23444-21apt-get08:51:072
344825213,22sleep23444-21apt-get08:51:072
1570325219,19sleep20-21swapper/207:46:132
2423625115,22sleep025884-21systemd-journal08:12:520
2423625115,22sleep025884-21systemd-journal08:12:520
199672517,19sleep00-21swapper/011:26:180
1952225112,20sleep219603-21mailstats09:41:192
133525112,24sleep01362-21irqstats12:11:130
109125115,17sleep20-21swapper/212:11:062
1770625013,19sleep117759-21irqstats09:36:131
1519725013,16sleep00-21swapper/007:45:540
1757224912,22sleep217576-21diskstats09:36:102
3087524814,19sleep00-21swapper/008:35:540
1021324811,22sleep010209-21apt-get10:56:050
250324711,20sleep22509-21diskstats10:31:112
92222459,24sleep013441-21snmpd07:26:120
92222459,24sleep013441-21snmpd07:26:120
213812446,16sleep10-21swapper/111:31:151
3125324311,20sleep131247-21apt-get08:36:071
3125324311,20sleep131247-21apt-get08:36:071
148282437,15sleep20-21swapper/209:26:212
36582428,14sleep10-21swapper/108:51:141
36582428,14sleep10-21swapper/108:51:141
242202409,17sleep10-21swapper/111:41:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional