You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-06 - 13:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Thu Feb 06, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15058995115,0cyclictest0-21swapper/119:34:151
1505899250,1cyclictest22633-21apt-get19:27:161
1505399250,25cyclictest31744-21apt-get22:07:150
1505999240,23cyclictest2420-21apt-get22:17:102
15058992422,2cyclictest20740-21apt-get23:00:021
15058992422,1cyclictest313-21systemd-journal19:20:001
1505899240,0cyclictest0-21swapper/120:22:161
1505399240,22cyclictest313-21systemd-journal20:35:000
1506099230,22cyclictest11726-40mandb00:00:003
1505999230,22cyclictest23460-21grep21:45:002
1505999230,22cyclictest17692-21idleruntime-cro22:55:012
1505899230,22cyclictest25289-21sh23:09:591
1505399230,22cyclictest24587-21sh19:30:010
15060992220,1cyclictest4842-21apt-get22:22:163
15060992220,1cyclictest0-21swapper/320:20:003
1506099220,2cyclictest3610-21awk21:00:003
1506099220,21cyclictest28312-21sendmail-msp00:40:013
1506099220,1cyclictest768-21runrttasks20:30:023
15059992222,0cyclictest0-21swapper/220:52:132
15059992220,1cyclictest14196-21apt00:04:592
15059992220,1cyclictest10794-21turbostat.cron21:15:002
1505999220,21cyclictest3616-21munin-run21:00:002
1505999220,21cyclictest28-21ksoftirqd/223:47:222
1505999220,21cyclictest15622-21turbostat.cron21:25:012
1505999220,21cyclictest14649-21dump-pmu-power22:45:002
1505999220,21cyclictest11574-21apt-get20:12:142
1505999220,1cyclictest612-21gdbus23:45:012
1505999220,1cyclictest20758-21perf20:30:012
1505999220,1cyclictest18375-21perf20:25:012
1505999220,1cyclictest0-21swapper/222:05:012
15058992220,1cyclictest25259-21awk00:30:011
1505899220,21cyclictest313-21systemd-journal23:00:001
1505899220,21cyclictest0-21swapper/122:47:191
1505899220,21cyclictest0-21swapper/122:47:181
1505899220,21cyclictest0-21swapper/122:17:101
1505899220,21cyclictest0-21swapper/121:12:171
1505899220,21cyclictest0-21swapper/121:00:001
1505899220,21cyclictest0-21swapper/119:12:131
1505899220,1cyclictest21079-21munin-run21:39:591
15053992221,1cyclictest0-21swapper/019:34:140
15053992220,1cyclictest9199-21idleruntime-cro22:30:000
1505399220,21cyclictest25961-21perf20:39:590
1505399220,21cyclictest23342-21apt-get23:07:160
1505399220,21cyclictest20938-21apt-get23:02:110
1505399220,21cyclictest19821-21awk19:20:010
1505399220,21cyclictest18787-21apt-get20:27:140
1505399220,21cyclictest14670-21perf22:45:000
15060992121,0cyclictest0-21swapper/322:37:103
15060992120,1cyclictest25741-21lspci19:34:143
15060992120,0cyclictest15424-21apt-get19:12:133
15060992119,1cyclictest17644-21cat00:15:003
1506099210,20cyclictest8830-21apt-get21:12:163
1506099210,20cyclictest0-21swapper/321:17:183
1506099210,1cyclictest22863-21perf00:25:003
1506099210,0cyclictest1874-21apt-get19:52:153
1506099210,0cyclictest0-21swapper/322:47:183
1506099210,0cyclictest0-21swapper/322:47:183
1506099210,0cyclictest0-21swapper/319:22:153
1505999218,12cyclictest28-21ksoftirqd/223:02:112
15059992121,0cyclictest0-21swapper/219:37:112
1505999210,20cyclictest0-21swapper/220:47:142
1505999210,20cyclictest0-21swapper/200:37:122
1505999210,1cyclictest22636-21chrt19:27:152
1505999210,0cyclictest28797-21apt-get23:22:162
1505999210,0cyclictest23342-21apt-get23:07:152
15058992121,0cyclictest0-21swapper/120:12:141
15058992120,0cyclictest15024-21apt-get00:07:141
15058992119,2cyclictest1445-21awk23:35:011
15058992119,1cyclictest28789-21apt-get20:47:141
15058992119,1cyclictest2010-21munin-run22:15:001
1505899210,21cyclictest0-21swapper/120:30:021
1505899210,20cyclictest6429-21apt-get21:07:151
1505899210,20cyclictest0-21swapper/120:02:141
15053992121,0cyclictest0-21swapper/021:12:160
15053992121,0cyclictest0-21swapper/020:07:150
15053992120,0cyclictest0-21swapper/000:02:170
1505399210,21cyclictest0-21swapper/019:52:140
1505399210,20cyclictest0-21swapper/022:57:110
1505399210,20cyclictest0-21swapper/021:07:150
1505399210,20cyclictest0-21swapper/020:22:150
1505399210,0cyclictest6724-21apt-get20:02:140
15060992020,0cyclictest0-21swapper/323:17:113
15060992020,0cyclictest0-21swapper/320:22:153
15060992019,0cyclictest26622-21apt-get19:37:103
15060992018,1cyclictest0-21swapper/320:50:003
1506099200,0cyclictest0-21swapper/322:42:123
1506099200,0cyclictest0-21swapper/300:02:163
15059992020,0cyclictest18126-21apt-get22:57:122
15059992019,1cyclictest271rcuc/219:14:592
15059992018,1cyclictest25935-21cron23:14:592
15059992018,1cyclictest15951-21ls20:20:002
15059992018,1cyclictest0-21swapper/221:57:102
15059992018,1cyclictest0-21swapper/221:57:102
15059992018,1cyclictest0-21swapper/220:44:592
1505999200,0cyclictest0-21swapper/219:42:152
15058992020,0cyclictest0-21swapper/123:57:111
15058992020,0cyclictest0-21swapper/122:37:111
15058992020,0cyclictest0-21swapper/119:42:141
15058992020,0cyclictest0-21swapper/100:17:121
15058992018,1cyclictest30754-21dump-pmu-power23:25:011
1505899200,0cyclictest0-21swapper/120:27:151
1505899200,0cyclictest0-21swapper/119:47:161
15053992020,0cyclictest0-21swapper/023:22:170
15053992020,0cyclictest0-21swapper/000:05:020
15053992019,1cyclictest0-21swapper/022:47:180
15053992019,1cyclictest0-21swapper/022:47:180
15053992017,2cyclictest593-21dbus-daemon22:05:000
1505399200,20cyclictest0-21swapper/020:47:130
1505399200,20cyclictest0-21swapper/000:15:020
1505399200,19cyclictest23897-21apt-get21:47:160
15060991919,0cyclictest0-21swapper/323:27:123
15060991919,0cyclictest0-21swapper/319:57:193
15060991919,0cyclictest0-21swapper/319:42:143
15060991919,0cyclictest0-21swapper/319:27:153
15060991917,1cyclictest25319-21idleruntime-cro23:10:003
15059991919,0cyclictest0-21swapper/221:27:142
15059991919,0cyclictest0-21swapper/200:15:002
15059991918,0cyclictest21579-21apt-get20:32:152
15058991919,0cyclictest0-21swapper/121:22:131
15058991918,1cyclictest4312-21apt-get19:57:181
15058991917,1cyclictest596-21in:imuxsock22:45:011
15053991919,0cyclictest0-21swapper/000:22:200
15060991818,0cyclictest8731-21perf20:05:013
15060991818,0cyclictest13641-21apt-get21:22:143
15060991818,0cyclictest0-21swapper/323:42:143
15060991818,0cyclictest0-21swapper/323:37:113
15060991818,0cyclictest0-21swapper/321:27:153
15060991818,0cyclictest0-21swapper/320:42:153
15060991818,0cyclictest0-21swapper/320:12:143
15060991818,0cyclictest0-21swapper/300:05:013
1506099180,17cyclictest0-21swapper/320:37:203
15059991818,0cyclictest0-21swapper/220:02:142
15059991816,1cyclictest0-21swapper/221:05:002
1505999180,1cyclictest6271-21sh19:59:592
15058991818,0cyclictest0-21swapper/121:42:131
15058991818,0cyclictest0-21swapper/121:27:141
15058991816,1cyclictest0-21swapper/123:50:001
1505899180,17cyclictest0-21swapper/120:17:201
1505899180,14cyclictest0-21swapper/120:52:171
15053991818,0cyclictest4288-21apt-get23:42:130
15053991818,0cyclictest10269-21apt-get22:37:110
15053991818,0cyclictest0-21swapper/023:57:100
15053991818,0cyclictest0-21swapper/021:42:140
15053991818,0cyclictest0-21swapper/019:22:160
15053991817,0cyclictest0-21swapper/022:22:150
15053991815,2cyclictest9854-21latency_hist22:35:010
15060991717,0cyclictest0-21swapper/323:54:593
15060991717,0cyclictest0-21swapper/322:17:113
15060991717,0cyclictest0-21swapper/322:07:173
15060991717,0cyclictest0-21swapper/322:02:153
15060991717,0cyclictest0-21swapper/300:27:153
15060991714,2cyclictest1175-21apt-config20:55:003
1506099170,16cyclictest1641-21cron20:10:003
15059991717,0cyclictest0-21swapper/221:37:102
15059991717,0cyclictest0-21swapper/219:47:172
15059991717,0cyclictest0-21swapper/200:07:152
15059991716,1cyclictest19793-21latency_hist19:20:002
15059991716,1cyclictest0-21swapper/222:47:192
15059991716,1cyclictest0-21swapper/222:47:192
15059991715,1cyclictest23897-21apt-get21:47:152
15059991714,2cyclictest25244-21ls00:30:012
1505999170,16cyclictest28-21ksoftirqd/222:07:172
15058991717,0cyclictest0-21swapper/123:42:141
15058991717,0cyclictest0-21swapper/121:47:161
15058991717,0cyclictest0-21swapper/120:37:191
15058991717,0cyclictest0-21swapper/100:37:101
15053991717,0cyclictest11574-21apt-get20:12:150
15053991717,0cyclictest0-21swapper/023:15:000
15053991717,0cyclictest0-21swapper/021:22:130
15053991717,0cyclictest0-21swapper/021:02:160
15053991717,0cyclictest0-21swapper/019:57:180
15053991715,1cyclictest773-21apt-get20:55:020
15053991715,1cyclictest4387-21systemctl22:19:590
15053991715,1cyclictest15968-21perf20:20:010
15060991616,0cyclictest0-21swapper/323:02:133
15060991616,0cyclictest0-21swapper/321:47:153
15060991613,2cyclictest32029-21cpuspeed_turbos19:47:173
1505999169,6cyclictest28-21ksoftirqd/222:27:202
15059991616,0cyclictest0-21swapper/223:37:122
15059991616,0cyclictest0-21swapper/222:22:162
15059991616,0cyclictest0-21swapper/221:17:192
15059991616,0cyclictest0-21swapper/221:07:142
1505999161,1cyclictest0-21swapper/220:07:152
1505999160,2cyclictest25675-21sed20:37:292
1505999160,11cyclictest28-21ksoftirqd/222:37:102
15058991616,0cyclictest0-21swapper/120:07:141
15058991616,0cyclictest0-21swapper/100:22:211
15058991614,1cyclictest26501-21turbostat21:55:011
1505899160,15cyclictest29023-21munin-plugin-st22:00:011
1505899160,15cyclictest29023-21munin-plugin-st22:00:011
1505899160,15cyclictest17652-21munin-run00:15:011
15053991616,0cyclictest26928-21apt-get21:57:100
15053991616,0cyclictest26928-21apt-get21:57:100
15053991616,0cyclictest0-21swapper/023:37:110
15053991616,0cyclictest0-21swapper/023:17:120
15053991615,0cyclictest19079-21apt-get21:37:110
15053991614,1cyclictest28617-21awk19:40:000
15053991614,1cyclictest0-21swapper/023:47:260
15053991614,1cyclictest0-21swapper/022:55:000
1505399160,1cyclictest1493-21/usr/sbin/munin19:49:590
1505399160,15cyclictest26520-21munin-run21:55:000
1505399160,15cyclictest0-21swapper/023:27:170
15060991515,0cyclictest0-21swapper/321:55:003
15060991515,0cyclictest0-21swapper/321:02:153
15060991513,1cyclictest0-21swapper/300:15:013
15060991512,2cyclictest313-21systemd-journal19:20:003
1506099150,6cyclictest0-21swapper/321:57:103
1506099150,6cyclictest0-21swapper/321:57:103
1506099150,2cyclictest8388-21cron21:10:003
1506099150,14cyclictest0-21swapper/322:35:003
15059991515,0cyclictest0-21swapper/219:52:152
15059991514,1cyclictest28-21ksoftirqd/219:25:002
15059991513,1cyclictest271rcuc/223:59:592
1505999150,1cyclictest0-21swapper/223:53:052
1505999150,14cyclictest0-21swapper/200:22:202
1505899158,6cyclictest18665-21basename21:34:591
15058991515,0cyclictest0-21swapper/122:27:151
15058991513,1cyclictest0-21swapper/119:37:131
1505899150,7cyclictest0-21swapper/100:02:161
15060991414,0cyclictest0-21swapper/321:37:103
15060991414,0cyclictest0-21swapper/320:25:453
1506099140,14cyclictest0-21swapper/323:33:263
1506099140,14cyclictest0-21swapper/323:22:573
1506099140,14cyclictest0-21swapper/322:58:233
1506099140,14cyclictest0-21swapper/322:25:073
1506099140,14cyclictest0-21swapper/321:42:193
1506099140,14cyclictest0-21swapper/321:33:083
1506099140,14cyclictest0-21swapper/300:30:103
1506099140,0cyclictest0-21swapper/323:10:553
1506099140,0cyclictest0-21swapper/322:51:003
1506099140,0cyclictest0-21swapper/322:11:153
15059991414,0cyclictest0-21swapper/221:31:032
1505999140,14cyclictest0-21swapper/223:17:332
1505999140,14cyclictest0-21swapper/222:32:142
1505999140,14cyclictest0-21swapper/222:15:002
1505999140,14cyclictest0-21swapper/219:30:302
1505999140,14cyclictest0-21swapper/200:17:312
1505999140,0cyclictest0-21swapper/223:33:582
1505999140,0cyclictest0-21swapper/221:50:412
15058991414,0cyclictest0-21swapper/122:50:061
15058991414,0cyclictest0-21swapper/122:05:121
15058991414,0cyclictest0-21swapper/122:00:081
15058991414,0cyclictest0-21swapper/119:52:141
1505899140,14cyclictest0-21swapper/123:15:451
1505899140,14cyclictest0-21swapper/123:10:101
1505899140,14cyclictest0-21swapper/122:21:121
1505899140,14cyclictest0-21swapper/121:00:021
1505899140,0cyclictest0-21swapper/121:16:331
15053991414,0cyclictest16043-21apt-get21:27:140
15053991414,0cyclictest0-21swapper/021:34:330
15053991414,0cyclictest0-21swapper/020:41:050
15053991414,0cyclictest0-21swapper/019:40:010
15053991414,0cyclictest0-21swapper/000:30:390
1505399140,14cyclictest0-21swapper/023:53:000
1505399140,14cyclictest0-21swapper/023:30:140
1505399140,14cyclictest0-21swapper/021:15:300
1505399140,14cyclictest0-21swapper/020:50:010
1505399140,14cyclictest0-21swapper/019:11:500
1505399140,14cyclictest0-21swapper/000:25:050
1505399140,0cyclictest0-21swapper/000:36:320
1505399140,0cyclictest0-21swapper/000:13:260
15059991313,0cyclictest0-21swapper/200:32:282
15058991313,0cyclictest0-21swapper/100:30:441
15053991313,0cyclictest0-21swapper/022:10:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional