You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-30 - 11:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s (updated Sat Sep 30, 2023 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13480276866,7646sleep21855-21kworker/2:220:19:342
25764262717,593sleep125767-21true19:17:261
21439231417,203sleep221445-21modprobe20:47:252
19933231414,280sleep220063-21grep22:23:052
13758230816,274sleep113897-21/usr/sbin/munin22:03:051
3195322790,15sleep00-21swapper/023:02:560
18067225143,90sleep218223-21latency_hist00:02:282
1705224158,167sleep21819-21latency_hist21:27:272
1705224158,167sleep21819-21latency_hist21:27:272
18402222982,128sleep20-21swapper/220:37:272
5871222046,104sleep25886-21modprobe19:57:252
25875222025,114sleep20-21swapper/222:43:002
25875222025,114sleep20-21swapper/222:43:002
3789220912,126sleep23821-21idleruntime-cro23:17:252
3789220912,126sleep23821-21idleruntime-cro23:17:252
24283219879,103sleep224369-21fschecks_time19:12:342
32067219117,87sleep20-21swapper/223:02:592
32618218027,115sleep232682-21cat21:22:272
30402217911,151sleep030555-21sed19:32:270
7183217310,149sleep27278-21switchtime21:43:022
1926421728,67sleep20-21swapper/220:38:082
27303216814,124sleep20-21swapper/221:03:042
27303216814,124sleep20-21swapper/221:03:042
3193216136,87sleep23205-21sh21:32:242
27578215414,126sleep327580-21sh00:32:263
21061215331,57sleep20-21swapper/222:27:532
2539621529,127sleep225510-21sed00:23:042
15053215266,17sleep20-21swapper/222:08:022
1356215260,54sleep20-21swapper/223:08:002
1356215260,54sleep20-21swapper/223:08:002
2220215116,119sleep12253-21sshd21:27:551
4737215010,126sleep24756-21idleruntime-cro21:37:252
4387215023,110sleep04425-21latency23:17:550
8427214445,18sleep20-21swapper/220:03:022
14924214447,61sleep214947-21latency_hist23:52:262
14924214447,61sleep214947-21latency_hist23:52:262
1550621385,11sleep10-21swapper/122:08:091
28321213756,60sleep20-21swapper/200:33:012
24557213640,22sleep20-21swapper/200:22:272
831213420,97sleep3895-21fschecks_time23:07:483
831213420,97sleep3895-21fschecks_time23:07:483
13720213418,102sleep00-21swapper/023:47:520
13720213418,102sleep00-21swapper/023:47:520
274521321,91sleep22935-21cmp19:47:272
14574213212,103sleep014643-21switchtime20:23:040
25210213026,89sleep225353-21sed22:42:272
814721297,32sleep20-21swapper/221:47:472
30251212917,98sleep330331-21tr21:13:033
28897212912,51sleep229044-21latency_hist19:27:282
22810212413,35sleep20-21swapper/222:33:022
26444212345,15sleep20-21swapper/219:18:032
19720212216,92sleep319783-21cut22:23:023
21199212134,64sleep021264-21grep19:03:000
11151212111,73sleep211359-21latency_hist21:57:282
23330212023,75sleep123397-21fschecks_time00:17:541
15023212010,76sleep2371-21sshd20:24:352
31968211917,89sleep331913-21/usr/sbin/munin19:37:273
31968211917,89sleep331913-21/usr/sbin/munin19:37:273
12160211733,62sleep20-21swapper/220:17:282
11514211719,81sleep111657-21unixbench_multi20:13:071
18945211663,14sleep20-21swapper/200:03:042
6354211531,71sleep26395-21latency_hist21:42:262
21287211512,85sleep021101-21apt-get00:12:270
11474211518,24sleep20-21swapper/221:57:442
23914211328,58sleep123908-21apt-get22:37:461
16714211324,68sleep30-21swapper/322:13:083
11035211330,45sleep20-21swapper/223:38:002
11035211330,45sleep20-21swapper/223:38:002
19844211219,77sleep119907-21/usr/sbin/munin00:07:481
18392211210,87sleep11-21systemd20:37:261
28566211110,85sleep128567-21sshd19:24:161
15543211127,60sleep20-21swapper/223:52:592
28579211020,30sleep10-21swapper/100:33:041
2944521097,88sleep21-21systemd21:12:252
2944521097,88sleep21-21systemd21:12:252
29926210855,14sleep20-21swapper/221:12:532
27638210845,18sleep20-21swapper/200:32:292
25070210849,18sleep20-21swapper/220:57:552
21063210826,72sleep2367-21snmpd00:12:052
856421056,51sleep20-21swapper/223:30:502
29221210512,75sleep329311-21perl19:27:543
27417210521,18sleep20-21swapper/222:47:572
1662921056,20sleep20-21swapper/220:30:252
12555210514,29sleep10-21swapper/123:43:001
23745210429,19sleep2286-21dbus-daemon19:12:262
29745210316,72sleep129789-21cpu21:12:481
31574210229,19sleep2367-21snmpd19:33:182
31574210229,19sleep2367-21snmpd19:33:182
16962210210,78sleep017013-21interrupts23:57:530
16535210223,58sleep20-21swapper/222:13:032
7526210123,53sleep17542-21fschecks_count23:27:501
14817210014,23sleep014862-21irqstats22:07:560
41152998,76sleep20-21swapper/223:17:452
1716629912,68sleep317210-21fschecks_count20:32:523
1622529821,55sleep016242-21fschecks_count22:12:570
636229716,18sleep00-21swapper/023:23:000
322142978,18sleep20-21swapper/219:37:462
322142978,18sleep20-21swapper/219:37:462
1588429714,60sleep316014-21latency_hist22:12:273
958829621,14sleep20-21swapper/223:33:042
41162960,8sleep30-21swapper/321:33:053
2768429631,46sleep327752-21fschecks_time19:22:533
1721529517,54sleep316727-21/usr/sbin/munin23:58:013
2543629412,67sleep325513-21switchtime20:58:043
975729312,27sleep20-21swapper/220:08:032
2938029321,14sleep20-21swapper/219:27:572
1641829325,52sleep316480-40mandb23:57:243
285392917,16sleep20-21swapper/222:52:452
665529021,14sleep20-21swapper/219:58:022
186292905,50sleep20-21swapper/222:18:122
1101428917,56sleep110525-21/usr/sbin/munin23:37:591
1101428917,56sleep110525-21/usr/sbin/munin23:37:591
68028818,20sleep30-21swapper/321:22:563
68028818,20sleep30-21swapper/321:22:563
75672866,20sleep00-21swapper/020:02:280
2523028523,28sleep10-21swapper/120:57:581
694228418,38sleep37084-21grep19:58:073
546228419,45sleep15503-21awk21:37:561
2703428417,51sleep027153-21timerwakeupswit21:03:020
2703428417,51sleep027153-21timerwakeupswit21:03:020
22822845,51sleep20-21swapper/219:43:562
58982837,36sleep20-21swapper/223:22:462
228402836,49sleep20-21swapper/200:16:502
2696028217,47sleep327031-21switchtime21:03:013
2696028217,47sleep327031-21switchtime21:03:013
2602228219,47sleep326095-21sed22:43:053
2602228219,47sleep326095-21sed22:43:053
1648428214,49sleep016504-21munin-run23:57:250
132182828,65sleep2367-21snmpd23:45:522
1902828123,30sleep10-21swapper/122:22:271
301152806,61sleep230111-21apt-get22:57:462
301152806,61sleep230111-21apt-get22:57:462
2986628015,47sleep329917-21latency_hist22:57:273
280712808,20sleep00-21swapper/022:50:090
2483728020,35sleep124870-21grep00:22:501
2295928014,49sleep122965-21ntp_states19:08:011
2107328029,30sleep20-21swapper/219:02:582
115328015,48sleep11203-21/usr/sbin/munin23:07:551
115328015,48sleep11203-21/usr/sbin/munin23:07:551
904127912,52sleep39106-21latency_hist20:07:283
3135827911,53sleep131499-21unixbench_singl19:33:051
3135827911,53sleep131499-21unixbench_singl19:33:051
217392798,32sleep20-21swapper/220:47:492
1093327927,34sleep310957-21munin-run21:57:253
240092786,12sleep20-21swapper/220:53:062
240092786,12sleep20-21swapper/220:53:062
896027714,49sleep38978-21apt-get23:32:443
2369927725,30sleep30-21swapper/322:37:273
2115827717,28sleep30-21swapper/300:12:263
186942776,56sleep00-21swapper/000:02:580
1293627721,16sleep00-21swapper/020:18:030
937527613,47sleep19430-21fschecks_count20:07:541
2022227613,50sleep1367-21snmpd22:23:531
2747827523,22sleep10-21swapper/122:48:001
2583127522,28sleep10-21swapper/122:42:591
2583127522,28sleep10-21swapper/122:42:591
2579927516,16sleep30-21swapper/319:17:283
1277627515,43sleep312381-21/usr/sbin/munin20:17:583
40627421,27sleep10-21swapper/121:22:461
40627421,27sleep10-21swapper/121:22:461
264902748,54sleep0367-21snmpd19:18:050
2011227419,39sleep020173-21/usr/sbin/munin00:07:550
1993727420,23sleep00-21swapper/020:42:260
1494827423,28sleep30-21swapper/323:52:273
1494827423,28sleep30-21swapper/323:52:273
1234827419,33sleep30-21swapper/323:42:543
2656827316,45sleep126655-21sed00:27:551
2330527320,24sleep00-21swapper/000:17:530
2072127328,26sleep30-21swapper/320:43:063
489127219,24sleep00-21swapper/019:52:550
157427214,43sleep31589-21uname19:42:563
1208127210,16sleep30-21swapper/320:17:253
795327124,22sleep30-21swapper/321:47:283
598327124,28sleep30-21swapper/323:22:513
1843327128,26sleep00-21swapper/022:18:060
142042717,29sleep20-21swapper/222:06:172
1280327118,37sleep112778-21latency_hist22:02:281
106652715,54sleep10-21swapper/121:53:561
1015927116,16sleep00-21swapper/021:53:020
950427015,25sleep10-21swapper/123:33:031
595127021,19sleep10-21swapper/119:57:281
2856827019,27sleep00-21swapper/021:07:590
2856827019,27sleep00-21swapper/021:07:590
1177227024,28sleep30-21swapper/321:57:583
3256526922,24sleep00-21swapper/021:22:240
2708326914,40sleep327108-21grep22:47:493
2494626919,38sleep024982-21fschecks_count20:57:530
2101526929,22sleep10-21swapper/119:02:571
976926815,37sleep09886-21users23:33:060
950126816,38sleep39558-21irqstats20:07:563
801526819,29sleep18077-21sshd20:02:531
318426818,28sleep00-21swapper/023:13:070
318426818,28sleep00-21swapper/023:13:070
2915826813,40sleep329222-21latency_hist00:37:263
2779326822,20sleep10-21swapper/121:07:271
2779326822,20sleep10-21swapper/121:07:271
2199426815,38sleep322000-21needreboot00:12:593
7982678,49sleep00-21swapper/019:39:460
7982678,49sleep00-21swapper/019:39:460
619826715,22sleep10-21swapper/119:57:481
2221826721,28sleep322237-21munin-run19:07:263
2080026722,26sleep00-21swapper/020:43:070
1876426718,29sleep318276-21/usr/sbin/munin00:03:013
1874126720,21sleep10-21swapper/120:37:551
1873326716,35sleep318737-21diskstats20:37:533
1660126714,31sleep116633-21/usr/sbin/munin22:13:061
819426615,22sleep10-21swapper/121:47:501
2551126614,36sleep324727-21/usr/sbin/munin00:23:053
2280426617,25sleep10-21swapper/122:33:011
1591126625,25sleep315950-21ntpq20:27:593
1346326612,38sleep313500-21ntp_states22:02:593
502726518,20sleep30-21swapper/319:52:583
2982826519,25sleep00-21swapper/021:12:510
211026516,26sleep30-21swapper/321:27:533
17682655,12sleep10-21swapper/119:43:021
1110526516,27sleep00-21swapper/020:12:580
2687326421,19sleep10-21swapper/119:18:111
2398326414,37sleep323991-21df22:37:483
1551626418,28sleep115534-21apt-get20:27:451
1304726418,27sleep013099-21cpuspeed22:02:470
369726321,18sleep00-21swapper/021:32:550
3097426313,34sleep331021-21users22:58:073
3097426313,34sleep331021-21users22:58:073
2775926314,29sleep027812-21/usr/sbin/munin19:22:540
1146226313,25sleep00-21swapper/021:57:380
231052627,11sleep00-21swapper/020:52:280
231052627,11sleep00-21swapper/020:52:280
2079226211,34sleep00-21swapper/022:27:430
1779426218,22sleep30-21swapper/322:17:533
1267726214,32sleep10-21swapper/120:17:561
61652616,13sleep00-21swapper/021:40:340
536226116,27sleep15410-21apt-config23:22:251
304726116,20sleep30-21swapper/319:47:373
2077026117,27sleep320788-21apt-get22:27:423
1529126120,16sleep30-21swapper/320:27:273
2379526025,18sleep30-21swapper/319:12:273
2276326019,24sleep00-21swapper/019:07:560
2030526021,24sleep10-21swapper/120:42:561
74325917,25sleep0739-21apt-get23:07:410
74325917,25sleep0739-21apt-get23:07:410
3174325917,26sleep331642-21/usr/sbin/munin23:02:523
2967725917,28sleep129745-21sed19:28:041
2603825916,24sleep00-21swapper/000:27:260
1873225918,23sleep118756-21ntp_states00:02:591
1116625910,21sleep00-21swapper/023:38:030
1116625910,21sleep00-21swapper/023:38:030
354625812,32sleep13552-21ntp_states19:48:031
2653425810,28sleep00-21swapper/000:27:540
256825820,23sleep00-21swapper/019:47:250
2602725713,26sleep326019-21apt-get19:17:523
1719125713,20sleep10-21swapper/123:57:591
926325621,18sleep00-21swapper/020:07:450
3072825618,19sleep10-21swapper/122:58:051
3072825618,19sleep10-21swapper/122:58:051
2906725616,16sleep10-21swapper/121:08:081
2906725616,16sleep10-21swapper/121:08:081
2820525612,27sleep328196-21apt-get21:07:493
2820525612,27sleep328196-21apt-get21:07:493
2149725612,16sleep10-21swapper/100:12:281
1431725619,20sleep10-21swapper/123:48:051
1431725619,20sleep10-21swapper/123:48:051
872425516,20sleep30-21swapper/321:48:043
3140425515,19sleep30-21swapper/321:17:513
3066425514,20sleep00-21swapper/022:58:040
3066425514,20sleep00-21swapper/022:58:040
2174025519,17sleep30-21swapper/320:47:503
1526225515,22sleep10-21swapper/123:52:541
868425416,17sleep30-21swapper/323:32:243
700725414,19sleep10-21swapper/121:42:571
549725416,20sleep30-21swapper/323:22:273
3229125412,25sleep132297-21diskstats19:37:491
3229125412,25sleep132297-21diskstats19:37:491
3197225415,18sleep00-21swapper/019:37:280
3197225415,18sleep00-21swapper/019:37:280
304425414,24sleep02561-21apt-get19:47:340
2854225416,23sleep028609-21df22:52:470
249722546,36sleep00-21swapper/022:39:590
2444325413,24sleep024438-21apt-get19:12:480
2100425412,22sleep10-21swapper/122:27:521
662725313,23sleep06678-21df21:42:480
357125312,16sleep10-21swapper/121:32:521
3146925312,23sleep131533-21fw_packets21:17:531
2263025313,24sleep122653-21users20:48:091
1708325313,24sleep017150-21df20:32:490
751925214,22sleep07525-21diskstats23:27:490
551625215,22sleep35522-21ntp_states21:37:593
3245725212,23sleep132567-21users23:03:061
2546225212,21sleep025529-21df22:42:500
2546225212,21sleep025529-21df22:42:500
1464425220,15sleep10-21swapper/120:23:051
1254825213,22sleep012554-21ntp_states23:42:590
880025113,18sleep00-21swapper/021:48:050
668625114,20sleep06753-21systemctl19:58:040
249125112,23sleep12014-21apt-get23:12:311
249125112,23sleep12014-21apt-get23:12:311
2249325114,22sleep022629-21sed20:48:080
589725013,23sleep15894-21apt-get23:22:451
3230425013,20sleep332395-21perl19:37:513
3230425013,20sleep332395-21perl19:37:513
2874125016,16sleep10-21swapper/122:52:521
2347225016,18sleep30-21swapper/320:52:553
2347225016,18sleep30-21swapper/320:52:553
1643825021,15sleep00-21swapper/020:28:080
2285624915,14sleep00-21swapper/022:33:040
29752488,21sleep30-21swapper/323:13:023
29752488,21sleep30-21swapper/323:13:023
284792489,16sleep00-21swapper/000:33:030
243892485,34sleep10-21swapper/120:55:321
243892485,34sleep10-21swapper/120:55:321
1947224811,20sleep019522-21irqstats22:22:550
2372224716,15sleep30-21swapper/300:18:043
38424410,16sleep00-21swapper/021:22:450
38424410,16sleep00-21swapper/021:22:450
195124412,16sleep00-21swapper/021:27:460
1052424212,16sleep30-21swapper/323:37:413
1052424212,16sleep30-21swapper/323:37:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional