You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-23 - 15:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s (updated Sat Sep 23, 2023 00:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
88252315410,3131sleep0195-21jbd2/mmcblk2p2-23:57:500
22432117315,1141sleep22244-21ntp_states21:53:242
3033529906,949sleep030378-21ntpq23:23:220
22526272913,702sleep122527-21sendmail-mta21:13:471
14655242130,355sleep014652-21ntp_states22:33:230
32143234310,319sleep332174-21ntpq23:28:333
253802321127,136sleep00-21swapper/023:08:150
2269623097,287sleep09951-1kworker/0:0H19:33:000
2269623097,287sleep09951-1kworker/0:0H19:33:000
22326229421,250sleep322420-21latency_hist19:32:513
22326229421,250sleep322420-21latency_hist19:32:513
1466227670,68sleep00-21swapper/020:08:170
1466227670,68sleep00-21swapper/020:08:170
31052271120,126sleep00-21swapper/021:57:500
1598822608,240sleep016078-21sed20:53:270
1767422389,167sleep017818-21unixbench_singl20:58:270
22861223753,70sleep00-21swapper/022:58:270
366822335,27sleep00-21swapper/023:40:050
16141223315,203sleep216174-21ntp_kernel_err22:38:222
25597223121,192sleep0227-21systemd-journal19:40:190
3167222275,15sleep00-21swapper/019:58:480
3167222275,15sleep00-21swapper/019:58:480
13420221935,62sleep00-21swapper/000:12:470
13420221935,62sleep00-21swapper/000:12:470
18394221828,58sleep00-21swapper/000:28:100
21351221711,183sleep020606-21/usr/sbin/munin22:53:300
25462221238,128sleep025602-21unixbench_singl21:23:310
20422221217,180sleep220461-21ntpq21:08:232
524722098,185sleep05389-21grep20:18:300
968922020,14sleep014119-21cyclictest20:33:270
968922020,14sleep014119-21cyclictest20:33:270
146022005,183sleep00-21swapper/021:52:290
20052218717,122sleep00-21swapper/019:23:310
20052218717,122sleep00-21swapper/019:23:310
26260218650,20sleep00-21swapper/019:43:180
7399218542,82sleep00-21swapper/023:52:530
2391218428,117sleep02437-21latency_hist23:37:510
20208218310,58sleep00-21swapper/000:33:190
20208218310,58sleep00-21swapper/000:33:190
498821809,40sleep00-21swapper/022:03:150
59621799,131sleep0774-21latency_hist23:32:500
30994217813,18sleep20-21swapper/219:58:182
30994217813,18sleep20-21swapper/219:58:182
16574216742,19sleep00-21swapper/019:13:260
8829216521,103sleep18838-21modprobe20:32:481
9945216415,130sleep09993-21ntpq22:18:250
9945216415,130sleep09993-21ntpq22:18:250
22693215911,132sleep222211-21apt-get19:32:582
22693215911,132sleep222211-21apt-get19:32:582
27553215638,60sleep00-21swapper/021:32:510
1165321555,17sleep00-21swapper/022:23:260
1544621549,130sleep015451-21grep00:18:290
28136214241,57sleep027603-21/usr/sbin/munin19:48:260
7586213826,79sleep00-21swapper/020:28:110
3129621388,78sleep00-21swapper/021:43:170
30412213720,95sleep2227-21systemd-journal19:57:492
17311213734,69sleep017372-21cpuspeed22:43:120
17311213734,69sleep017372-21cpuspeed22:43:120
1072121377,89sleep00-21swapper/000:03:190
1886921358,91sleep018912-21tr21:03:200
4170213419,93sleep34217-21awk20:17:493
11996213417,96sleep30-21swapper/320:42:523
604221339,82sleep00-21swapper/020:23:110
2316921337,16sleep00-21swapper/021:18:100
1439021277,105sleep00-21swapper/000:13:280
13424212711,102sleep11-21systemd00:12:491
13424212711,102sleep11-21systemd00:12:491
2952721245,28sleep00-21swapper/023:21:070
343021229,76sleep03516-21mailstats20:13:300
9382212015,92sleep39396-21kernelversion23:58:463
9382212015,92sleep39396-21kernelversion23:58:463
3101211913,88sleep33133-21proc_pri23:38:273
9870211821,75sleep110007-21sed23:58:561
9870211821,75sleep110007-21sed23:58:561
28432211811,87sleep328530-21/usr/sbin/munin19:48:293
1577021158,68sleep00-21swapper/022:38:120
2940211414,77sleep23007-21/usr/sbin/munin23:38:222
2760221146,87sleep10-21swapper/119:48:071
26583211429,28sleep20-21swapper/223:12:502
28475211310,60sleep00-21swapper/021:33:280
28388211325,44sleep30-21swapper/323:18:083
11110211335,37sleep00-21swapper/020:38:290
21963211232,62sleep021568-21/usr/sbin/munin21:13:210
3599211013,82sleep23715-21cat20:13:322
30672211018,75sleep230765-21apt-config21:42:502
30672211018,75sleep230765-21apt-config21:42:502
979210920,67sleep11112-21head21:48:311
3034321096,18sleep00-21swapper/019:56:540
13461210919,55sleep00-21swapper/020:47:290
5303210819,56sleep00-21swapper/023:46:150
2734121087,63sleep027427-21mailstats23:13:240
26177210821,68sleep326257-21fschecks_time19:43:173
8938210718,70sleep38814-40mandb23:57:533
21759210758,32sleep21-21systemd22:56:182
2666210616,71sleep10-21swapper/120:12:501
2666210616,71sleep10-21swapper/120:12:501
21674210625,40sleep00-21swapper/019:28:360
21674210625,40sleep00-21swapper/019:28:360
5161210516,17sleep30-21swapper/322:03:193
14338210516,75sleep214412-21mailstats20:48:282
1028210519,65sleep1537-21apt-get23:33:001
6446210423,40sleep00-21swapper/022:08:120
14077210418,62sleep314114-21grep20:48:213
2957721039,77sleep129633-21/usr/sbin/munin21:38:171
2957721039,77sleep129633-21/usr/sbin/munin21:38:171
15352210316,71sleep115354-21sshd19:09:391
13945210134,45sleep00-21swapper/022:32:510
1434921009,19sleep014119-21cyclictest19:08:090
665229918,66sleep26745-21timerandwakeup20:23:292
5142998,54sleep0664-21sed20:03:280
5142998,54sleep0664-21sed20:03:280
2988929913,64sleep330026-21/usr/sbin/munin19:53:303
2533429915,69sleep325460-21cat21:23:303
2005229935,16sleep00-21swapper/021:08:120
172362996,12sleep20-21swapper/200:23:312
1607929913,59sleep116229-21sed20:53:281
1077629931,44sleep110808-21uniq00:03:321
2949029822,57sleep00-21swapper/021:38:160
2949029822,57sleep00-21swapper/021:38:160
2063129831,41sleep120625-21apt-get22:53:111
502629716,63sleep15106-21fschecks_time22:03:161
3082129717,63sleep330843-21fschecks_count19:58:143
3082129717,63sleep330843-21fschecks_count19:58:143
1378629716,62sleep313692-21/usr/sbin/munin00:13:113
718229617,51sleep17313-21grep22:08:291
1812929620,15sleep00-21swapper/000:27:490
1447929630,46sleep00-21swapper/020:48:300
380029532,19sleep00-21swapper/021:58:230
1277729526,28sleep20-21swapper/219:03:152
1235029515,65sleep212387-21basename22:27:482
958329416,15sleep00-21swapper/023:58:510
958329416,15sleep00-21swapper/023:58:510
1851529321,26sleep30-21swapper/300:28:173
2130329216,62sleep221320-21latency_hist00:37:572
2130329216,62sleep221320-21latency_hist00:37:572
283762910,7sleep30-21swapper/321:33:263
2405029121,12sleep00-21swapper/023:03:190
465529017,17sleep10-21swapper/122:02:521
2480229018,51sleep224819-21fschecks_count21:23:162
308892885,12sleep10-21swapper/123:23:471
1980428821,43sleep319784-21latency_hist21:07:513
3252828715,55sleep232557-21/usr/sbin/munin20:03:182
3252828715,55sleep232557-21/usr/sbin/munin20:03:182
1974528623,46sleep119788-21munin-plugin-st21:07:491
1509328523,42sleep215236-21unixbench_singl19:08:272
321192849,13sleep20-21swapper/223:28:322
1354928428,29sleep30-21swapper/320:47:523
3247428211,55sleep232624-21latency_hist21:47:512
2471628211,52sleep10-21swapper/121:23:111
196328214,55sleep32007-21basename21:53:173
1537228210,61sleep3367-21snmpd22:36:533
908028118,28sleep00-21swapper/022:15:430
178712818,32sleep00-21swapper/019:18:100
263412798,12sleep30-21swapper/321:28:143
111627913,50sleep17694-1kworker/1:1H20:07:491
111627913,50sleep17694-1kworker/1:1H20:07:491
1845027726,34sleep318526-21wc19:18:293
1754627724,35sleep317565-21apt19:17:493
224172760,10sleep20-21swapper/222:58:152
2167827617,40sleep321769-21fschecks_time21:13:163
806927528,24sleep20-21swapper/222:13:152
2235227522,22sleep10-21swapper/122:58:151
2004427515,28sleep00-21swapper/022:49:020
2004427515,28sleep00-21swapper/022:49:020
949827419,36sleep29527-21/usr/sbin/munin20:33:222
949827419,36sleep29527-21/usr/sbin/munin20:33:222
1371827413,17sleep20-21swapper/200:13:102
750627312,23sleep10-21swapper/123:53:141
2446827319,22sleep10-21swapper/121:22:511
1859427318,41sleep118651-21cat22:47:501
1859427318,41sleep118651-21cat22:47:501
1703327222,35sleep317050-21fschecks_count20:58:143
1577527212,40sleep215355-21/usr/sbin/munin20:53:212
1406727219,31sleep114086-21munin-run19:07:501
103927218,38sleep31040-21sshd20:07:193
103927218,38sleep31040-21sshd20:07:193
3165527120,36sleep10-21swapper/121:43:261
164827124,27sleep31708-21sed20:08:203
164827124,27sleep31708-21sed20:08:203
1628827118,32sleep30-21swapper/322:38:273
1326227125,20sleep30-21swapper/319:03:283
375327018,26sleep30-21swapper/321:58:223
2970727016,22sleep20-21swapper/223:22:512
25812709,47sleep22582-21sshd20:12:182
25812709,47sleep22582-21sshd20:12:182
2072827025,27sleep10-21swapper/121:08:301
1771127017,24sleep20-21swapper/222:43:232
1771127017,24sleep20-21swapper/222:43:232
1441327018,32sleep114563-21sed00:13:291
993526921,32sleep39943-21/usr/sbin/munin22:18:243
993526921,32sleep39943-21/usr/sbin/munin22:18:243
2508926911,44sleep225156-21swap19:38:272
197852699,13sleep10-21swapper/119:23:241
197852699,13sleep10-21swapper/119:23:241
1486126925,22sleep10-21swapper/122:33:281
1233426915,35sleep312377-21fschecks_time00:08:183
1233426915,35sleep312377-21fschecks_time00:08:183
425026825,19sleep10-21swapper/120:17:511
238626815,33sleep12459-21/usr/sbin/munin21:53:271
1994026821,24sleep10-21swapper/100:33:111
1994026821,24sleep10-21swapper/100:33:111
1916626822,29sleep119255-21ls19:22:521
1849826821,25sleep20-21swapper/221:03:102
1697926821,26sleep10-21swapper/119:13:351
833426719,24sleep30-21swapper/320:28:303
3029726717,36sleep130298-21sshd19:56:191
2751126621,25sleep20-21swapper/223:13:262
229002667,13sleep10-21swapper/119:33:211
229002667,13sleep10-21swapper/119:33:211
1809526617,27sleep30-21swapper/300:27:483
1606826616,20sleep20-21swapper/219:12:552
1428126622,24sleep20-21swapper/222:33:132
132622666,48sleep10-21swapper/120:44:561
93726514,22sleep20-21swapper/223:32:522
579626516,18sleep10-21swapper/120:22:511
215002657,12sleep20-21swapper/219:28:342
215002657,12sleep20-21swapper/219:28:342
1137926518,21sleep20-21swapper/220:38:352
2499526415,25sleep10-21swapper/123:07:481
1701126425,25sleep117045-21tune2fs00:23:261
1320626421,26sleep0367-21snmpd19:03:250
1273626419,23sleep20-21swapper/222:28:162
1240826418,23sleep10-21swapper/122:27:511
2998126324,18sleep30-21swapper/321:38:273
2998126324,18sleep30-21swapper/321:38:273
2246826322,21sleep30-21swapper/322:58:173
1949726313,35sleep219068-21/usr/sbin/munin22:48:272
1949726313,35sleep219068-21/usr/sbin/munin22:48:272
1353626319,27sleep20-21swapper/220:47:512
1148526322,21sleep30-21swapper/322:23:213
919826214,32sleep39205-21df_inode20:33:153
919826214,32sleep39205-21df_inode20:33:153
2462826213,33sleep324640-21fschecks_count19:38:163
2343326212,25sleep30-21swapper/321:18:173
2144626217,27sleep121499-21/usr/sbin/munin19:28:331
2144626217,27sleep121499-21/usr/sbin/munin19:28:331
2038526211,37sleep320386-21sshd19:24:193
2038526211,37sleep320386-21sshd19:24:193
159162629,36sleep315986-21sensors_temp20:53:263
1523826215,18sleep30-21swapper/319:08:283
1265326217,30sleep312702-21/usr/sbin/munin22:28:133
868326114,25sleep30-21swapper/322:13:283
1353026123,21sleep20-21swapper/200:12:582
1353026123,21sleep20-21swapper/200:12:582
611126022,19sleep30-21swapper/320:23:143
477126028,18sleep20-21swapper/220:18:202
3120026024,19sleep10-21swapper/119:58:251
3120026024,19sleep10-21swapper/119:58:251
3069726021,17sleep30-21swapper/323:23:293
2830526015,25sleep20-21swapper/221:33:262
2653826015,32sleep126582-21systemd-run23:12:491
2416026012,33sleep324275-21cat19:37:513
2416026012,33sleep324275-21cat19:37:513
1782026011,34sleep117851-21users20:58:281
798325921,18sleep10-21swapper/122:13:101
786225918,23sleep27956-21memory20:28:212
2625525915,26sleep226251-21apt-get21:28:092
2504625918,26sleep325125-21date23:07:503
1583325914,29sleep315907-21cat00:18:403
557925814,26sleep30-21swapper/323:47:513
3022725818,22sleep230320-21load23:23:202
2374625818,23sleep223742-21apt-get23:03:092
1876225816,22sleep30-21swapper/322:47:523
85962575,11sleep20-21swapper/223:55:502
543925715,22sleep10-21swapper/123:47:491
489925717,23sleep24896-21apt-get22:03:092
2028725717,17sleep30-21swapper/321:08:193
1990325719,15sleep20-21swapper/219:23:292
1990325719,15sleep20-21swapper/219:23:292
1692125714,22sleep20-21swapper/220:58:052
1578625722,20sleep10-21swapper/100:18:381
1272925714,26sleep112735-21diskstats22:28:151
1128625712,26sleep10-21swapper/120:38:341
940725614,23sleep19497-21memory20:33:211
940725614,23sleep19497-21memory20:33:211
774125614,18sleep30-21swapper/322:12:513
596125616,24sleep35956-21apt-get23:48:123
269182566,32sleep10-21swapper/123:13:131
1865625615,23sleep20-21swapper/200:28:202
1577125614,25sleep115765-21apt-get22:38:131
926125518,20sleep20-21swapper/223:58:402
926125518,20sleep20-21swapper/223:58:402
549025514,24sleep20-21swapper/223:47:502
449525514,24sleep34492-21apt-get20:18:093
321325516,19sleep10-21swapper/123:38:291
2909325518,17sleep10-21swapper/123:18:301
2605925514,21sleep20-21swapper/219:43:072
250272556,12sleep10-21swapper/119:38:261
414825415,21sleep24253-21users21:58:292
148325411,17sleep30-21swapper/321:52:493
601825313,24sleep16036-21apt-get20:23:071
3169525313,23sleep131212-21apt-get23:27:591
2826725316,18sleep10-21swapper/121:33:251
2672625319,16sleep10-21swapper/121:28:241
2529225312,24sleep325288-21apt-get23:08:103
2016125316,20sleep320206-21irqstats00:33:173
2016125316,20sleep320206-21irqstats00:33:173
1420725317,16sleep10-21swapper/120:48:231
1087425316,21sleep20-21swapper/222:22:512
1087425316,21sleep20-21swapper/222:22:512
623825216,16sleep20-21swapper/223:48:232
3182725212,26sleep30-21swapper/321:43:283
2072625212,24sleep30-21swapper/322:53:173
2762325113,22sleep227621-21apt-get19:48:082
2690425114,18sleep30-21swapper/323:13:113
262982518,18sleep10-21swapper/119:43:191
1904625112,22sleep118568-21apt-get22:47:541
1904625112,22sleep118568-21apt-get22:47:541
1757725114,22sleep317593-21kernelversion22:43:193
1189625116,17sleep20-21swapper/200:07:492
773825021,16sleep20-21swapper/222:12:502
2202425012,22sleep222048-21ntp_states21:13:232
105732504,11sleep30-21swapper/300:02:533
2323824912,16sleep20-21swapper/221:18:122
153824811,16sleep20-21swapper/221:52:522
649524715,14sleep10-21swapper/123:48:301
1828424713,17sleep20-21swapper/219:18:242
1909324613,16sleep30-21swapper/322:48:123
1909324613,16sleep30-21swapper/322:48:123
95652459,18sleep10-21swapper/122:18:101
95652459,18sleep10-21swapper/122:18:101
1983024213,13sleep10-21swapper/100:32:591
153082429,16sleep20-21swapper/200:18:102
106523910,14sleep30-21swapper/323:33:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional