You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-03-01 - 01:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s (updated Thu Feb 29, 2024 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10588243649,371sleep010724-21grep10:55:190
10588243649,371sleep010724-21grep10:55:190
270692423113,191sleep027083-21fschecks_count11:50:000
25307236014,302sleep025373-21fschecks_time10:00:020
3537234624,296sleep23670-21grep12:15:142
19920230015,261sleep019958-21fschecks_count09:45:000
1825622990,25sleep118257-21kernelversion07:55:061
31721226724,228sleep231732-21uname10:20:032
17955224837,28sleep018013-21sendmail09:39:300
31906223240,38sleep00-21swapper/012:04:580
7176222947,78sleep07184-21modprobe12:29:300
5221222421,40sleep00-21swapper/010:39:530
79922216,192sleep00-21swapper/010:25:020
2016222176,198sleep00-21swapper/008:00:150
9214221640,55sleep00-21swapper/009:10:060
7753221513,187sleep07358-21/usr/sbin/munin09:05:100
1824722157,195sleep018255-21grep07:55:060
3033922067,65sleep030398-21ps10:15:090
205222065,29sleep00-21swapper/010:29:390
4974220110,125sleep00-21swapper/010:39:320
3000121950,9sleep00-21swapper/011:59:320
2788219110,169sleep02794-21grep12:14:560
966218518,147sleep3713-21apt-get12:09:343
2462521848,53sleep00-21swapper/009:59:130
71292178115,42sleep00-21swapper/009:04:330
71292178115,42sleep00-21swapper/009:04:330
291521770,28sleep22919-21unixbench_multi10:30:122
1181217716,86sleep31194-21fschecks_count08:44:573
4856216921,15sleep00-21swapper/007:14:350
21300216910,58sleep00-21swapper/008:05:040
14366216934,94sleep014412-21apt-config07:44:320
14366216934,94sleep014412-21apt-config07:44:320
8866216626,114sleep28952-21wc12:34:332
12329216527,123sleep011494-21/usr/sbin/munin07:35:270
5433216127,120sleep15483-21if_eth010:40:021
2760221589,134sleep027676-21latency08:25:030
319215211,124sleep0472-21latency_hist10:24:340
3352215111,124sleep23397-21grep07:10:052
16570214816,24sleep30-21swapper/311:15:173
12967214614,51sleep00-21swapper/009:20:160
16181214528,73sleep016271-21latency_hist07:49:330
3872214412,114sleep03954-21grep07:10:190
4359213941,37sleep00-21swapper/012:19:530
31669213332,46sleep0372-21snmpd08:39:290
7838213212,104sleep37845-21sh09:05:133
18166213228,91sleep018246-21grep11:20:110
18166213228,91sleep018246-21grep11:20:110
30679213012,59sleep030751-21switchtime12:00:100
10100213011,59sleep00-21swapper/007:30:060
7077212916,99sleep27102-21/usr/sbin/munin09:04:312
7077212916,99sleep27102-21/usr/sbin/munin09:04:312
3016721276,15sleep00-21swapper/008:34:320
21794212717,91sleep321951-21unixbench_multi08:05:153
21719212315,93sleep121750-21fschecks_count09:50:061
23160212124,69sleep20-21swapper/208:10:112
23330211938,22sleep00-21swapper/008:10:130
2393211818,65sleep00-21swapper/008:49:300
1424211558,15sleep00-21swapper/012:10:030
1483821145,27sleep00-21swapper/009:29:130
76021120,6sleep10-21swapper/112:09:311
1609521129,16sleep00-21swapper/011:14:580
9566211121,27sleep09591-21modprobe10:54:310
9566211121,27sleep09591-21modprobe10:54:310
20580211112,78sleep120719-21latency_hist11:29:331
1034721115,27sleep00-21swapper/012:37:260
30927210921,73sleep231001-21/usr/sbin/munin08:35:112
28527210929,40sleep028569-21if_eth010:10:000
28527210929,40sleep028569-21if_eth010:10:000
23495210914,79sleep223506-21fschecks_count09:55:032
2774210824,62sleep12911-21grep10:30:121
15840210718,74sleep215808-21latency_hist11:14:322
3734210519,71sleep33779-21fschecks_count10:34:593
1758621057,63sleep0372-21snmpd07:53:200
6876210427,16sleep00-21swapper/007:19:520
31270210413,75sleep1231-21systemd-journal12:03:221
18879210446,20sleep00-21swapper/009:40:110
11493210412,74sleep111501-21/usr/sbin/munin07:34:521
28978210216,69sleep229047-21fschecks_time08:30:022
15494210114,35sleep00-21swapper/011:10:420
32401210045,16sleep00-21swapper/008:40:050
2206821009,52sleep022172-21latency_hist11:34:330
2708129925,22sleep20-21swapper/208:24:322
1346429922,18sleep00-21swapper/011:05:090
277212985,11sleep00-21swapper/010:05:440
277212985,11sleep00-21swapper/010:05:440
2398229834,43sleep00-21swapper/011:40:020
2033229816,70sleep00-21swapper/011:27:460
85022976,13sleep00-21swapper/012:31:400
535329718,68sleep00-21swapper/008:57:420
3100229713,66sleep331070-21/usr/sbin/munin08:35:123
308329416,63sleep33023-21latency_hist07:09:333
241732947,76sleep0372-21snmpd08:14:490
77362937,32sleep00-21swapper/010:45:540
2300929317,53sleep123157-21unixbench_multi11:35:111
1828429317,63sleep318331-21grep09:39:543
1486229320,56sleep214893-21awk09:29:312
107562935,29sleep00-21swapper/009:15:070
3208829028,31sleep20-21swapper/208:39:592
1776429021,53sleep217848-21fschecks_time11:20:012
1776429021,53sleep217848-21fschecks_time11:20:012
95992895,32sleep00-21swapper/007:28:370
95992895,32sleep00-21swapper/007:28:370
822528915,57sleep18357-21cut07:24:341
3119828913,61sleep331227-21latency_hist10:19:323
2464828916,52sleep124712-21http09:59:311
36972885,29sleep00-21swapper/008:51:160
36972885,29sleep00-21swapper/008:51:160
2023828818,46sleep120379-21unixbench_multi08:00:161
125042875,29sleep00-21swapper/011:02:060
125042875,29sleep00-21swapper/011:02:060
895328623,38sleep39164-21latency_hist12:34:343
267542865,27sleep00-21swapper/008:20:590
2580628612,57sleep325876-21sensors_temp10:00:463
232392865,28sleep00-21swapper/011:35:220
2604428518,52sleep326122-21grep11:45:143
2080628519,50sleep320830-21kworker/u8:208:04:303
996228425,26sleep30-21swapper/310:55:063
996228425,26sleep30-21swapper/310:55:063
511528417,54sleep15151-21fschecks_time07:14:401
3141328426,40sleep231539-21latency_hist12:04:322
2888128412,14sleep10-21swapper/108:29:571
1590128419,42sleep116030-21grep09:30:111
2302428314,54sleep123067-21tr08:10:061
224528337,29sleep20-21swapper/207:05:212
111828222,20sleep10-21swapper/110:25:111
943128121,41sleep29490-21sendmail_mailqu09:10:162
63832810,6sleep20-21swapper/212:25:072
2537628112,53sleep125430-21/usr/sbin/munin10:00:031
385828013,49sleep13906-21/usr/sbin/munin10:35:011
1491728022,39sleep114885-21latency_hist09:29:341
394827921,38sleep30-21swapper/308:54:303
394827921,38sleep30-21swapper/308:54:303
877727820,32sleep10-21swapper/112:34:321
806927815,18sleep30-21swapper/310:49:333
77842787,14sleep10-21swapper/109:05:111
2664927818,27sleep00-21swapper/011:49:070
199627819,42sleep02023-21/usr/sbin/munin07:05:030
315062777,14sleep10-21swapper/108:37:241
3112027716,43sleep131143-21modprobe10:19:301
2262527718,27sleep00-21swapper/009:51:260
157862777,11sleep00-21swapper/009:30:090
812527611,49sleep28165-21sh07:24:312
140552756,11sleep20-21swapper/207:40:502
140552756,11sleep20-21swapper/207:40:502
2885127412,42sleep128921-21sensors_temp10:10:091
2885127412,42sleep128921-21sensors_temp10:10:091
2553027423,27sleep30-21swapper/308:19:333
1730927416,33sleep317385-21threads09:35:093
157192746,56sleep30-21swapper/311:13:363
926727321,30sleep19262-21apt-get12:34:561
485627315,29sleep20-21swapper/212:20:112
2879927313,46sleep328825-21kernelversion11:55:043
2868227317,41sleep328700-21netstat10:10:033
2868227317,41sleep328700-21netstat10:10:033
99212727,10sleep10-21swapper/109:11:081
736427219,27sleep30-21swapper/307:20:123
489427223,34sleep24935-21awk10:39:302
300127221,32sleep13029-21munin-run07:09:311
245322728,10sleep30-21swapper/309:58:033
1958127224,22sleep10-21swapper/109:44:321
1897327222,25sleep10-21swapper/111:24:321
1897327222,25sleep10-21swapper/111:24:321
1061827214,27sleep30-21swapper/307:30:173
619627118,29sleep10-21swapper/112:25:031
3257727120,36sleep132653-21ls08:40:111
146627116,37sleep11072-21/usr/sbin/munin08:45:031
1434127115,29sleep30-21swapper/307:44:313
1434127115,29sleep30-21swapper/307:44:313
1223927122,25sleep20-21swapper/211:00:182
1223927122,25sleep20-21swapper/211:00:182
3154227020,32sleep331748-21gpgv12:04:333
279952705,54sleep30-21swapper/310:09:143
279952705,54sleep30-21swapper/310:09:143
2735227015,37sleep227362-21/usr/sbin/munin11:50:062
2416427021,25sleep30-21swapper/311:40:053
214027014,40sleep12152-21vmstat12:10:161
1475127019,38sleep314755-21gpgv07:44:353
129592706,11sleep20-21swapper/211:04:422
2708626918,26sleep10-21swapper/111:50:011
549426824,28sleep25518-21munin-run08:59:302
54102685,53sleep10-21swapper/108:58:241
2995126812,43sleep229971-21grep10:14:592
104052688,44sleep30-21swapper/312:38:133
726826715,33sleep17328-21grep10:45:191
515226719,25sleep20-21swapper/207:14:412
485526722,23sleep30-21swapper/308:55:113
2364026718,25sleep20-21swapper/211:39:322
142562678,43sleep314368-21cat09:25:123
2292426628,16sleep30-21swapper/308:10:043
189382664,51sleep20-21swapper/207:55:532
1564526621,20sleep20-21swapper/209:30:072
1225026621,22sleep20-21swapper/209:20:022
68622656,47sleep30-21swapper/309:01:453
68622656,47sleep30-21swapper/309:01:453
580726515,22sleep10-21swapper/108:59:531
580726515,22sleep10-21swapper/108:59:531
296326514,22sleep20-21swapper/208:50:012
296326514,22sleep20-21swapper/208:50:012
2801226522,20sleep10-21swapper/108:25:121
262626511,34sleep32695-21sensors_temp10:30:103
2291426518,25sleep30-21swapper/309:54:323
199526521,26sleep20-21swapper/208:45:122
121212659,13sleep20-21swapper/207:35:252
1060626521,24sleep20-21swapper/212:39:342
1801526416,31sleep118033-21munin-run09:39:311
1695426410,38sleep216977-21irqstats09:35:002
1441326422,21sleep10-21swapper/107:44:331
1441326422,21sleep10-21swapper/107:44:331
353326320,26sleep30-21swapper/307:10:103
2857526318,25sleep30-21swapper/308:29:303
2179926312,39sleep20-21swapper/211:31:442
1784926319,26sleep317900-21sort11:20:023
1784926319,26sleep317900-21sort11:20:023
1571826323,18sleep30-21swapper/309:30:083
1190626312,36sleep111909-21sshd11:00:131
1190626312,36sleep111909-21sshd11:00:131
1057626321,23sleep10-21swapper/109:15:041
909926217,22sleep30-21swapper/307:25:123
909926217,22sleep30-21swapper/307:25:123
295532626,47sleep10-21swapper/111:56:391
259102627,11sleep10-21swapper/111:45:121
195942629,32sleep219622-21/usr/sbin/munin11:25:062
1943526215,21sleep30-21swapper/311:25:033
1930226217,30sleep319436-21latency_hist07:59:333
1454126218,21sleep20-21swapper/207:44:342
651426113,32sleep36540-21cat10:44:323
2136326123,16sleep20-21swapper/209:49:342
2038126122,24sleep220446-21swap09:45:142
1922626111,34sleep119289-21/usr/sbin/munin11:24:571
1281326119,26sleep312960-21awk09:20:153
1122826121,21sleep10-21swapper/110:59:321
1088626112,23sleep30-21swapper/309:15:103
1024326122,24sleep20-21swapper/207:30:082
279626011,35sleep12859-21df12:14:571
181426024,19sleep20-21swapper/212:10:132
1273526013,28sleep30-21swapper/311:04:313
1273526013,28sleep30-21swapper/311:04:313
104872608,22sleep20-21swapper/209:14:572
507925922,16sleep10-21swapper/112:20:141
3206525921,21sleep30-21swapper/310:20:133
2258125914,28sleep322640-21/usr/sbin/munin11:35:003
1875225915,19sleep20-21swapper/209:40:092
1742325913,23sleep20-21swapper/211:19:332
1673025916,23sleep30-21swapper/307:50:043
873925813,17sleep30-21swapper/310:50:093
873925813,17sleep30-21swapper/310:50:093
54225815,25sleep3536-21apt-get10:24:533
2840125823,22sleep228451-21grep10:09:582
2840125823,22sleep228451-21grep10:09:582
162722585,11sleep10-21swapper/107:49:341
1501025821,18sleep10-21swapper/107:45:061
628925717,25sleep26290-21sshd10:42:132
3190925713,26sleep231892-21apt-get12:05:002
2925025712,21sleep30-21swapper/308:30:073
1983725713,26sleep10-21swapper/109:44:571
2473325617,16sleep20-21swapper/208:15:112
2389325615,24sleep123886-21apt-get11:39:551
2123625613,27sleep321256-21/usr/sbin/munin11:30:123
2111025613,26sleep121105-21apt-get08:04:571
2107725614,28sleep321126-21apt-config09:49:313
1625225616,24sleep116071-21/usr/sbin/munin11:15:071
1299725612,26sleep313064-21df11:04:563
119125623,18sleep20-21swapper/210:25:122
1087525616,19sleep10-21swapper/107:30:191
887025512,20sleep20-21swapper/207:25:072
887025512,20sleep20-21swapper/207:25:072
841925519,17sleep10-21swapper/110:50:031
841925519,17sleep10-21swapper/110:50:031
832025514,24sleep28312-21apt-get10:49:562
832025514,24sleep28312-21apt-get10:49:562
3152825517,20sleep10-21swapper/110:19:581
2652125517,23sleep226595-21latency_hist10:04:332
705025419,22sleep20-21swapper/210:45:122
550725417,19sleep30-21swapper/307:15:073
435225413,25sleep30-21swapper/312:19:523
278625412,24sleep32781-21apt-get12:14:553
2599725417,17sleep30-21swapper/308:20:053
2587025412,17sleep20-21swapper/211:45:092
1954825411,27sleep219545-21apt-get07:59:532
121725410,25sleep31235-21apt-get12:09:543
334125312,24sleep13335-21apt-get07:09:561
2685825315,22sleep226859-21diskstats10:05:012
2685825315,22sleep226859-21diskstats10:05:012
882425216,20sleep18869-21grep07:25:061
882425216,20sleep18869-21grep07:25:061
2624225215,21sleep126248-21ntp_states08:20:111
2243225216,20sleep222562-21latency_hist08:09:342
1030625214,21sleep210329-21ntp_states10:55:132
1030625214,21sleep210329-21ntp_states10:55:132
894125116,17sleep30-21swapper/309:09:573
86425112,16sleep30-21swapper/308:44:333
735725112,25sleep27365-21acpi09:04:552
295225113,23sleep12962-21irqstats08:50:001
295225113,23sleep12962-21irqstats08:50:001
1336325117,16sleep10-21swapper/109:24:331
1299525111,23sleep112990-21apt-get11:04:531
2943225016,19sleep20-21swapper/211:55:152
2389425014,17sleep20-21swapper/211:39:562
1642125010,16sleep20-21swapper/207:49:512
1461825013,23sleep114625-21df_inode11:10:021
629524813,21sleep35902-21/usr/sbin/munin12:25:053
2745324814,15sleep30-21swapper/311:50:113
1008524810,18sleep10-21swapper/110:55:081
1158824612,20sleep311594-21df_inode07:34:583
2677324511,17sleep10-21swapper/110:04:561
2677324511,17sleep10-21swapper/110:04:561
2427524515,19sleep124177-21/usr/sbin/munin08:14:581
2732924111,15sleep20-21swapper/208:24:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional