You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 12:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed Apr 29, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
253982233015,2302sleep123737-21kworker/1:223:39:071
253982233015,2302sleep123737-21kworker/1:223:39:071
1374027316,713sleep313763-21ntpq23:00:543
32516250110,475sleep232577-21/usr/sbin/munin00:01:002
2535124080,8sleep10-21swapper/120:11:111
239223946,351sleep32428-21unin-run00:10:303
19649237813,351sleep319675-21latency_hist19:55:313
4813232915,298sleep24806-21apt-get20:50:462
2317923180,11sleep222646-21/usr/sbin/munin23:30:592
2317923180,11sleep222646-21/usr/sbin/munin23:30:592
319162310127,166sleep331521-21/usr/sbin/munin22:15:563
319162310127,166sleep331521-21/usr/sbin/munin22:15:563
23584230311,278sleep323476-21/usr/sbin/munin21:50:483
23584230311,278sleep323476-21/usr/sbin/munin21:50:483
13116226837,33sleep313254-21latency_hist19:35:323
18079225315,214sleep118221-21latency_hist19:50:321
23916224711,222sleep213441-21snmpd21:50:552
23916224711,222sleep213441-21snmpd21:50:552
27937224213,216sleep327976-21cat22:05:303
7218224016,211sleep27242-21grep19:15:482
3081622330,6sleep30-21swapper/320:30:313
21747223114,203sleep321836-21/usr/sbin/munin20:00:523
24638222772,72sleep324313-21apt-get20:10:333
21189222421,187sleep121224-21awk20:00:301
26489222315,194sleep226546-21fschecks_time20:15:492
991222139,182sleep310021-21apt-key22:50:323
991222139,182sleep310021-21apt-key22:50:323
16004221214,173sleep316026-21kernelversion21:25:533
16004221214,173sleep316026-21kernelversion21:25:533
8042221112,171sleep18197-21unixbench_multi00:26:021
5573220935,158sleep35661-21latency_hist00:20:313
7484218412,23sleep37604-21sed22:41:003
29491218421,142sleep30-21swapper/322:10:313
29491218421,142sleep30-21swapper/322:10:313
23260218115,148sleep223295-21date21:50:312
4019217912,155sleep34051-21latency_hist00:15:303
6478217532,132sleep36528-21fschecks_count20:55:523
18399217511,148sleep218474-21fschecks_time19:50:502
39932172126,22sleep30-21swapper/319:05:563
28685217213,123sleep328834-21latency_hist23:50:313
5326216855,29sleep35327-21cpu22:35:473
9122216615,14sleep30-21swapper/322:46:043
8908216555,30sleep38915-21wc00:30:443
6160216313,136sleep36308-21date20:55:323
27701216115,133sleep327725-21cut20:20:303
3275216019,123sleep13334-21unin-run19:05:301
3275216019,123sleep13334-21unin-run19:05:301
1122216014,30sleep20-21swapper/200:05:442
20128215417,53sleep320201-21ls23:21:003
10801215412,101sleep310875-21cat19:25:593
32300215119,36sleep032326-21munin-run20:35:300
3184521459,96sleep331988-21cut00:00:323
25446214539,19sleep30-21swapper/321:55:533
23000214525,106sleep30-21swapper/320:05:413
23000214525,106sleep30-21swapper/320:05:413
9954214417,110sleep30-21swapper/321:05:583
13897214327,97sleep013912-21sh21:20:290
13897214327,97sleep013912-21sh21:20:290
32701214210,86sleep30-21swapper/320:35:493
1609021418,100sleep313441-21snmpd19:44:373
143221417,117sleep01542-21users22:21:020
12401214170,57sleep312512-21latency_hist21:15:313
12401214170,57sleep312512-21latency_hist21:15:313
1774421399,95sleep317778-21ps21:30:583
6554213814,88sleep10-21swapper/100:21:011
17625213812,112sleep01-21systemd19:46:190
2807213712,91sleep30-21swapper/322:26:023
8982213411,78sleep30-21swapper/319:20:543
21018213023,23sleep30-21swapper/321:40:593
22196212816,92sleep022340-21grep20:01:010
3104721240,10sleep131033-21munin-run22:15:291
29497212113,13sleep30-21swapper/323:51:003
16525211810,15sleep00-21swapper/023:10:480
6628211711,91sleep30-21swapper/319:13:073
15260211516,23sleep315279-21netstat23:05:543
3097521135,11sleep00-21swapper/022:15:130
29580211311,88sleep029658-21grep20:25:490
12468211317,77sleep212626-21unixbench_multi22:56:002
1708211210,16sleep30-21swapper/320:40:463
2040721118,90sleep020418-21grep21:40:440
714521087,20sleep37294-21sed00:25:313
7358210710,16sleep30-21swapper/300:25:443
1178921077,14sleep30-21swapper/322:55:453
17710210616,23sleep10-21swapper/123:15:291
7500210512,74sleep30-21swapper/319:15:533
64221049,14sleep30-21swapper/322:20:453
24811210421,65sleep024313-21apt-get20:10:460
24222210312,37sleep30-21swapper/323:35:423
24222210312,37sleep30-21swapper/323:35:423
5630210112,66sleep25522-21/usr/sbin/munin19:10:482
1937121017,13sleep30-21swapper/323:20:323
1305421017,14sleep30-21swapper/321:15:553
1305421017,14sleep30-21swapper/321:15:553
31230210014,68sleep131372-21/usr/sbin/munin23:56:021
15526210015,67sleep215675-21latency_hist21:25:312
2608029927,45sleep226101-21modprobe20:15:292
171442997,15sleep30-21swapper/319:45:583
142672995,36sleep30-21swapper/319:36:053
83982986,35sleep30-21swapper/321:00:573
279992986,12sleep30-21swapper/320:20:473
2179029815,68sleep021907-21cat23:26:020
142602987,15sleep30-21swapper/321:20:483
142602987,15sleep30-21swapper/321:20:483
2826929715,17sleep10-21swapper/122:05:491
2826929715,17sleep10-21swapper/122:05:491
1153929713,67sleep011586-21latency_hist19:30:320
1129529716,61sleep111030-21/usr/sbin/munin21:10:521
1129529716,61sleep111030-21/usr/sbin/munin21:10:521
312842967,13sleep30-21swapper/320:30:523
1997229610,12sleep30-21swapper/319:55:473
2433029415,63sleep124420-21latency_hist20:10:311
2433029415,63sleep124420-21latency_hist20:10:311
500729316,58sleep25068-21sendmail-msp22:35:292
296992937,11sleep30-21swapper/322:10:393
1498029321,49sleep214997-21uniq23:05:492
237522927,12sleep30-21swapper/323:33:113
237522927,12sleep30-21swapper/323:33:113
211242929,31sleep30-21swapper/323:25:453
1977329253,19sleep30-21swapper/321:36:063
6982905,12sleep30-21swapper/300:03:483
326529035,33sleep03270-21modprobe19:05:290
326529035,33sleep03270-21modprobe19:05:290
3014829022,49sleep030194-40mandb23:55:290
268832906,10sleep30-21swapper/323:43:173
230832907,10sleep30-21swapper/321:48:503
174952905,11sleep30-21swapper/323:12:453
49382896,11sleep30-21swapper/322:34:383
539528810,62sleep15487-21threads20:51:021
1156728818,55sleep111712-21latency_hist22:55:311
1156728818,55sleep111712-21latency_hist22:55:311
43882875,11sleep30-21swapper/320:48:453
30528719,42sleep1453-21grep00:01:031
283242860,6sleep20-21swapper/222:05:492
283242860,6sleep20-21swapper/222:05:492
1785828520,51sleep117925-21switchtime21:31:001
589628414,48sleep05909-21fschecks_count00:20:480
42702846,60sleep10-21swapper/119:06:021
2400928414,53sleep124139-21latency_hist23:35:321
2400928414,53sleep124139-21latency_hist23:35:321
1032228420,38sleep110416-21sort19:25:491
1225528311,19sleep00-21swapper/019:31:010
572928221,33sleep00-21swapper/019:10:500
45352829,59sleep04627-21/usr/sbin/munin00:15:520
3109328219,34sleep20-21swapper/222:15:312
1498328214,52sleep114988-21/usr/sbin/munin19:40:471
692628123,39sleep10-21swapper/120:56:021
618227918,45sleep26230-21sed00:20:542
39727920,35sleep1481-21latency_hist22:20:311
39727920,35sleep1481-21latency_hist22:20:311
340727920,41sleep23544-21head00:11:012
1231527919,23sleep00-21swapper/022:55:580
1342927821,31sleep10-21swapper/123:00:481
2639427718,29sleep00-21swapper/022:00:310
12742764,16sleep00-21swapper/020:38:370
270727534,20sleep30-21swapper/319:01:003
270727534,20sleep30-21swapper/319:01:003
490127416,24sleep20-21swapper/219:10:312
405327423,27sleep00-21swapper/000:15:310
2647427419,18sleep00-21swapper/020:15:470
196922745,53sleep10-21swapper/123:20:501
1861927413,19sleep00-21swapper/021:35:310
1620427412,47sleep016291-21latency_hist23:10:320
1461327415,40sleep014644-21munin-run19:40:300
728727311,14sleep20-21swapper/222:40:542
684427317,35sleep26363-21/usr/sbin/munin20:55:592
2439427317,21sleep00-21swapper/023:35:510
2439427317,21sleep00-21swapper/023:35:510
2112727315,29sleep10-21swapper/123:25:471
997527212,33sleep10-21swapper/121:06:001
66427225,19sleep10-21swapper/120:35:591
2006127229,20sleep20-21swapper/223:20:592
1842727211,43sleep018474-21ntpq23:15:530
1380827221,27sleep20-21swapper/223:00:572
1764727123,27sleep20-21swapper/221:30:522
1563927119,27sleep20-21swapper/219:41:002
374127012,29sleep20-21swapper/220:45:582
3137327015,40sleep230451-21/usr/sbin/munin23:56:032
28312706,12sleep10-21swapper/120:43:471
2329427019,23sleep20-21swapper/220:05:522
2329427019,23sleep20-21swapper/220:05:522
1802527022,20sleep20-21swapper/223:15:422
501326916,38sleep15158-21/usr/sbin/munin00:16:011
288602695,13sleep10-21swapper/120:21:051
74142686,13sleep00-21swapper/020:57:230
1967826818,22sleep00-21swapper/019:55:320
1644826813,15sleep20-21swapper/219:45:332
1966126714,36sleep019690-21fschecks_count23:20:490
1957726715,37sleep119641-21modprobe19:55:291
1546926717,25sleep10-21swapper/123:06:011
372526616,32sleep13740-21/usr/sbin/munin20:45:571
2624726614,22sleep00-21swapper/023:40:560
1239026613,33sleep20-21swapper/219:31:032
905126517,18sleep00-21swapper/019:20:550
840626514,29sleep00-21swapper/021:00:580
386126520,25sleep03881-21sed19:05:530
126926512,37sleep01333-21/usr/sbin/munin00:05:500
1042226521,26sleep20-21swapper/222:51:202
1042226521,26sleep20-21swapper/222:51:202
979226420,22sleep20-21swapper/222:50:312
3045026422,22sleep00-21swapper/023:55:410
2789326427,21sleep2281-21dbus-daemon22:05:292
260726425,18sleep20-21swapper/219:00:592
260726425,18sleep20-21swapper/219:00:592
2253526412,21sleep00-21swapper/021:46:010
1549026419,21sleep10-21swapper/121:25:301
1549026419,21sleep10-21swapper/121:25:301
141726422,19sleep20-21swapper/220:40:282
1310026411,36sleep113120-21/usr/sbin/munin21:15:571
1310026411,36sleep113120-21/usr/sbin/munin21:15:571
113626411,34sleep1616-21/usr/sbin/munin22:20:591
94792637,17sleep00-21swapper/021:05:380
753226325,23sleep00-21swapper/019:15:540
1829326316,25sleep10-21swapper/123:15:501
159126319,24sleep10-21swapper/100:06:001
136272638,46sleep20-21swapper/221:16:592
136272638,46sleep20-21swapper/221:16:592
972026211,27sleep10-21swapper/122:50:301
882626217,28sleep00-21swapper/022:45:570
326472620,7sleep00-21swapper/000:01:010
315532628,37sleep00-21swapper/022:15:450
315532628,37sleep00-21swapper/022:15:450
2204026216,26sleep10-21swapper/121:45:501
2088826218,23sleep10-21swapper/121:40:571
1472526215,24sleep00-21swapper/019:40:340
66542616,13sleep10-21swapper/119:13:251
262662616,12sleep20-21swapper/223:40:582
1016926116,25sleep00-21swapper/000:35:290
38562605,45sleep10-21swapper/100:14:191
284642605,11sleep00-21swapper/023:48:430
2736826010,35sleep127415-21df23:45:451
2086026011,36sleep013441-21snmpd19:56:370
940625921,21sleep10-21swapper/100:30:581
684625915,26sleep06865-21apt-get22:40:400
497725911,34sleep04978-21sshd22:35:070
307272595,34sleep10-21swapper/120:30:101
2794925911,27sleep20-21swapper/220:20:462
2027225920,19sleep20-21swapper/219:55:542
179125916,28sleep21797-21cat20:40:492
1142825917,19sleep00-21swapper/021:10:550
1142825917,19sleep00-21swapper/021:10:550
2681925814,19sleep00-21swapper/022:00:510
219625813,27sleep02266-21df22:25:460
167312585,17sleep00-21swapper/021:27:070
167312585,17sleep00-21swapper/021:27:070
782725716,22sleep20-21swapper/200:25:592
463225713,25sleep00-21swapper/020:50:320
426125715,25sleep24258-21apt-get00:15:412
420125717,24sleep14207-21ntp_states22:30:561
3238025719,18sleep20-21swapper/220:35:322
2878525712,20sleep00-21swapper/022:06:000
2878525712,20sleep00-21swapper/022:06:000
2569225716,21sleep20-21swapper/221:56:002
2387025715,25sleep123915-21/usr/sbin/munin21:50:541
2387025715,25sleep123915-21/usr/sbin/munin21:50:541
2140225717,27sleep215670-21sshd21:42:072
1946225712,27sleep219539-21grep21:36:012
1390025716,20sleep00-21swapper/023:00:590
796425614,22sleep00-21swapper/000:26:010
30172568,24sleep00-21swapper/020:45:310
2925725615,21sleep10-21swapper/123:50:541
2422625616,20sleep00-21swapper/021:51:020
2422625616,20sleep00-21swapper/021:51:020
2011525615,26sleep120117-21cron21:40:291
1612725620,19sleep20-21swapper/221:25:562
1612725620,19sleep20-21swapper/221:25:562
1150825616,23sleep211514-21ntp_states21:10:582
1150825616,23sleep211514-21ntp_states21:10:582
3179625514,22sleep10-21swapper/120:31:031
2930425513,17sleep20-21swapper/223:50:552
262422557,12sleep10-21swapper/120:15:321
2227425514,20sleep20-21swapper/223:29:072
2152725515,21sleep10-21swapper/120:00:461
1704425516,23sleep117017-21latency_hist21:30:311
1704425516,23sleep117017-21latency_hist21:30:311
103662556,38sleep00-21swapper/022:51:140
103662556,38sleep00-21swapper/022:51:140
267825418,20sleep10-21swapper/122:26:001
263102544,40sleep10-21swapper/122:00:031
2579425418,18sleep10-21swapper/123:40:441
1436925410,20sleep20-21swapper/221:20:512
974425314,17sleep20-21swapper/221:05:532
921025310,18sleep10-21swapper/119:21:001
876325313,25sleep28767-21diskstats19:20:492
775025312,23sleep17773-21users22:41:021
3079525312,22sleep20-21swapper/220:30:312
2550425317,21sleep225552-21unin-run23:40:292
2550425317,21sleep225552-21unin-run23:40:292
219125312,24sleep22185-21apt-get22:25:442
1310725317,16sleep10-21swapper/123:00:311
129982534,10sleep10-21swapper/119:35:021
544925213,15sleep00-21swapper/022:35:520
540825213,19sleep20-21swapper/222:35:502
3151425211,23sleep231015-21apt-get22:15:352
3151425211,23sleep231015-21apt-get22:15:352
2428425214,22sleep01-21systemd20:10:300
2428425214,22sleep01-21systemd20:10:300
1024825214,17sleep20-21swapper/200:35:322
2278825117,15sleep20-21swapper/220:05:312
1415525111,22sleep114255-21users19:36:031
64025010,23sleep2636-21apt-get22:20:442
1692225017,16sleep20-21swapper/223:10:582
1479825014,23sleep014916-21timerandwakeup21:21:010
1479825014,23sleep014916-21timerandwakeup21:21:010
844324913,23sleep18470-21cat19:20:321
38712499,18sleep00-21swapper/020:46:000
799624815,16sleep10-21swapper/121:00:471
1885824811,21sleep018854-21apt-get21:35:430
1356524811,19sleep20-21swapper/219:35:512
837324710,16sleep20-21swapper/221:00:562
2313024715,14sleep00-21swapper/023:30:580
2313024715,14sleep00-21swapper/023:30:580
274162469,14sleep20-21swapper/223:45:462
2802524513,18sleep00-21swapper/020:20:480
275372455,27sleep10-21swapper/122:01:041
553424412,16sleep10-21swapper/122:35:531
1026924310,20sleep210302-21df19:25:462
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional