You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-25 - 10:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat May 25, 2024 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18641210288,9sleep30-21swapper/319:06:533
18853210187,9sleep10-21swapper/119:09:361
1887829472,16sleep00-21swapper/019:09:540
1880728564,17sleep20-21swapper/219:09:012
18970993126,5cyclictest0-21swapper/221:27:412
1897099270,27cyclictest0-21swapper/200:00:012
1897299240,23cyclictest0-21swapper/321:20:163
1897299240,1cyclictest0-21swapper/322:43:363
1897299230,22cyclictest0-21swapper/323:24:443
18959992319,3cyclictest13-21ksoftirqd/023:20:180
144742230,0sleep00-21swapper/000:16:400
1897299226,6cyclictest496-21dbus-daemon19:30:043
18970992218,3cyclictest0-21swapper/223:15:152
1896699211,19cyclictest19530-21ssh21:35:241
1897299200,20cyclictest0-21swapper/323:16:573
1896699200,1cyclictest0-21swapper/121:44:301
1895999201,3cyclictest0-21swapper/021:15:120
1895999201,3cyclictest0-21swapper/021:15:120
18970991918,1cyclictest0-21swapper/221:45:142
18970991915,2cyclictest0-21swapper/223:55:262
1897099190,19cyclictest0-21swapper/221:16:222
1897099190,19cyclictest0-21swapper/221:16:212
1897099180,0cyclictest0-21swapper/222:09:352
18966991816,1cyclictest0-21swapper/123:30:111
18966991812,5cyclictest25-21ksoftirqd/120:30:181
1897299179,2cyclictest3273-21cut00:35:183
1897299176,8cyclictest0-21swapper/320:55:003
18972991710,2cyclictest1881-21chrt22:10:213
18970991713,3cyclictest24519-21apt-get19:20:002
1897099170,15cyclictest0-21swapper/222:13:412
1896699170,1cyclictest0-21swapper/122:58:051
1896699170,16cyclictest0-21swapper/100:05:141
1895999174,7cyclictest5181-21awk19:45:000
18959991711,4cyclictest18956-21cyclictest23:05:010
1895999171,11cyclictest0-21swapper/021:50:150
1897299169,2cyclictest25736-21head19:20:173
1897299169,2cyclictest25736-21head19:20:173
1897299167,2cyclictest224-21systemd-journal23:46:193
1897299160,16cyclictest0-21swapper/322:49:273
1897299160,0cyclictest0-21swapper/300:18:263
1897099169,2cyclictest0-21swapper/200:13:242
1897099161,13cyclictest0-21swapper/221:21:582
1897099160,15cyclictest0-21swapper/221:35:192
1897099160,0cyclictest0-21swapper/223:05:522
1895999160,1cyclictest0-21swapper/022:58:090
1897299151,2cyclictest0-21swapper/321:18:123
1897299151,2cyclictest0-21swapper/321:18:123
18972991512,3cyclictest0-21swapper/322:58:053
1897099153,6cyclictest1-21systemd19:15:002
18970991513,1cyclictest14067-21diskmemload21:41:062
1897099150,15cyclictest25351-21ntp_states00:25:242
1897099150,13cyclictest0-21swapper/221:59:372
1896699159,4cyclictest19373-21cron22:00:001
1896699154,1cyclictest0-21swapper/122:22:401
1896699151,2cyclictest0-21swapper/120:50:141
18959991510,4cyclictest5280-21tr23:00:190
1895999150,8cyclictest29815-21perf00:30:000
1895999150,14cyclictest0-21swapper/023:50:220
1895999150,14cyclictest0-21swapper/022:50:130
1897299145,5cyclictest224-21systemd-journal21:30:003
18972991410,2cyclictest27657-21systemd-cgroups19:25:043
1897299140,1cyclictest0-21swapper/300:21:413
1897099144,5cyclictest224-21systemd-journal19:23:012
1897099144,5cyclictest224-21systemd-journal19:23:012
1897099144,3cyclictest224-21systemd-journal19:40:002
1897099141,12cyclictest1164-21ssh00:32:222
1897099141,12cyclictest0-21swapper/223:44:532
18970991410,3cyclictest10447-21cat21:00:262
1897099140,14cyclictest0-21swapper/200:21:302
1896699146,6cyclictest12067-21cups-browsed20:15:301
1896699142,7cyclictest12374-21/usr/sbin/munin23:50:361
1896699141,8cyclictest18768-21/usr/sbin/munin23:10:251
1896699141,12cyclictest30564-21kill19:30:041
1896699141,11cyclictest20843-21sed19:10:361
18966991410,3cyclictest12708-21wc00:15:231
1896699140,12cyclictest18161-21cut20:10:201
1895999149,4cyclictest8457-21ssh23:25:180
18959991413,1cyclictest13-21ksoftirqd/023:35:000
1897299139,3cyclictest2596-21systemd19:38:103
1897299139,3cyclictest19778-21ssh22:23:313
1897299134,7cyclictest32428-21/usr/sbin/munin00:05:263
18972991313,0cyclictest0-21swapper/321:52:353
18972991310,2cyclictest359-21grep21:45:203
18972991310,2cyclictest24213-21sh19:19:083
1897299130,13cyclictest0-21swapper/300:03:583
1897299130,12cyclictest31348-21wc00:30:233
1897099139,3cyclictest14342-21sh22:20:002
1897099138,4cyclictest5980-21wc20:50:362
1897099137,4cyclictest16828-21tail22:20:362
1897099134,5cyclictest224-21systemd-journal20:45:002
18970991310,2cyclictest22680-21grep23:35:172
18970991310,2cyclictest14734-21sh20:05:002
1897099130,1cyclictest0-21swapper/223:45:172
1897099130,1cyclictest0-21swapper/200:17:242
1896699139,3cyclictest31596-21cron20:40:001
1896699139,3cyclictest13578-21grep21:05:331
1896699139,2cyclictest22883-21tail20:20:211
1896699139,2cyclictest22805-21cut23:35:171
1896699137,5cyclictest0-21swapper/122:35:471
1896699133,3cyclictest224-21systemd-journal19:17:161
18966991311,1cyclictest0-21swapper/123:59:081
18966991310,2cyclictest9923-21sleep19:55:011
18966991310,2cyclictest6520-21sh21:50:161
18966991310,2cyclictest6487-21sed23:00:341
18966991310,2cyclictest290432sleep122:30:141
18966991310,2cyclictest26019-21tr00:00:281
18966991310,2cyclictest23365-21wc21:15:231
18966991310,2cyclictest23365-21wc21:15:231
18966991310,2cyclictest18394-21awk23:10:001
18966991310,2cyclictest17775-21grep00:20:151
18966991310,2cyclictest17127-21perf20:10:001
18966991310,2cyclictest10524-21head21:00:261
1896699130,6cyclictest30145-21latency_hist22:55:011
1896699130,1cyclictest0-21swapper/123:42:531
1895999139,3cyclictest8188-21ssh21:27:370
1895999139,3cyclictest23509-21sh19:15:400
1895999139,2cyclictest3694-21ssh23:45:170
1895999137,3cyclictest29876-21sshd19:28:140
1895999133,1cyclictest141rcu_preempt22:27:420
1895999132,3cyclictest30541-21sshd19:30:040
18959991310,2cyclictest9941-21grep22:15:350
18959991310,2cyclictest5955-21dump-pmu-power21:50:010
18959991310,2cyclictest29163-21cut22:30:150
18959991310,2cyclictest25266-21grep00:00:200
18959991310,2cyclictest13814-21cut21:55:180
18959991310,2cyclictest11068-21sh00:15:000
1895999130,10cyclictest0-21swapper/023:40:180
1897299129,2cyclictest5730-21cut23:00:233
1897299129,2cyclictest4393-21grep22:35:263
1897299129,2cyclictest24282-21basename20:25:003
1897299129,2cyclictest18867-21cat20:10:333
1897299122,4cyclictest0-21swapper/319:45:113
18972991210,1cyclictest14060-21latency_hist21:10:003
1897299120,6cyclictest0-21swapper/323:10:063
1897299120,0cyclictest0-21swapper/323:30:223
1897099129,2cyclictest32083-21head00:05:222
1897099129,2cyclictest15640-21kworker/2:122:50:102
1897099128,3cyclictest16949-21grep23:30:312
1897099128,2cyclictest12678-21head21:05:202
1897099125,3cyclictest8109-21cat21:50:332
1897099124,4cyclictest224-21systemd-journal19:26:232
1897099121,8cyclictest7056-21sed23:00:392
1897099121,1cyclictest37850irq/131-snd_hda22:58:292
18970991210,1cyclictest15613-21cut20:05:222
1897099120,2cyclictest0-21swapper/223:51:202
1897099120,1cyclictest0-21swapper/222:44:162
1897099120,1cyclictest0-21swapper/221:30:142
1897099120,11cyclictest23886-21grep20:20:362
1896699129,2cyclictest9115-21cron20:59:591
1896699129,2cyclictest5269-21sort21:25:251
1896699129,2cyclictest10275-21grep19:55:131
1896699128,3cyclictest3543-21cron21:25:001
1896699128,3cyclictest28251-21wc22:05:271
1896699128,3cyclictest26845-21cron20:30:001
1896699128,3cyclictest14538-21idleruntime-cro23:30:001
1896699128,3cyclictest13232-21tail21:30:361
1896699126,2cyclictest224-21systemd-journal19:26:161
1896699125,3cyclictest224-21systemd-journal21:15:001
1896699125,3cyclictest224-21systemd-journal21:15:001
1896699123,5cyclictest493-21systemd-logind19:23:501
1896699123,5cyclictest493-21systemd-logind19:23:501
18966991210,1cyclictest0-21swapper/122:44:361
1895999129,2cyclictest7728-21grep20:55:220
1895999129,2cyclictest27362-21sed22:05:160
1895999129,2cyclictest19096-21grep21:35:200
1895999129,2cyclictest15407-21tail21:10:210
1895999129,2cyclictest15407-21tail21:10:200
1895999128,3cyclictest32730-21cut20:40:190
1895999128,3cyclictest29778-21grep20:35:170
1895999128,2cyclictest27000-21wc20:30:000
1895999126,5cyclictest28343-21ssh21:41:470
1895999126,5cyclictest0-21swapper/022:47:350
1895999120,9cyclictest0-21swapper/023:11:520
1897299119,1cyclictest17944-21cat21:35:013
1897299118,2cyclictest9658-21tr21:00:183
1897299118,2cyclictest8099-21wc19:50:143
1897299118,2cyclictest5674-21expr00:10:163
1897299118,2cyclictest4148-21chrt20:49:593
1897299118,2cyclictest32470-21ssh22:32:333
1897299118,2cyclictest32246-21ssh22:09:043
1897299118,2cyclictest29258-21cut23:40:163
1897299118,2cyclictest28542-21expr20:30:283
1897299118,2cyclictest26753-21ssh00:26:263
1897299118,2cyclictest24650-21cat20:25:003
1897299118,2cyclictest23633-21grep22:25:353
1897299118,2cyclictest23541-21sh22:50:123
1897299118,2cyclictest23054-21cat23:35:213
1897299118,2cyclictest20437-21ssh21:35:393
1897299118,2cyclictest20038-21cut22:00:143
1897299118,2cyclictest18960-21rm23:55:273
1897299118,2cyclictest17455-21ssh23:09:013
1897299118,2cyclictest14024-21basename21:55:203
1897299118,2cyclictest11608-21ssh23:27:133
1897299118,2cyclictest10746-21grep23:50:163
1897299116,2cyclictest224-21systemd-journal19:14:583
1897299115,2cyclictest224-21systemd-journal20:40:003
1897299112,8cyclictest8693-21/usr/sbin/munin22:15:203
1897299111,6cyclictest0-21swapper/320:40:023
1897099119,1cyclictest11403-21head19:55:242
1897099118,2cyclictest9194-21sh23:25:252
1897099118,2cyclictest7313-21sh00:39:012
1897099118,2cyclictest4067-21chrt20:48:572
1897099118,2cyclictest2682-21sh23:20:412
1897099118,2cyclictest25516-21head20:25:232
1897099118,2cyclictest20870-21sh22:00:222
1897099118,2cyclictest20671-21cat20:15:232
1897099118,2cyclictest14961-21expr21:10:172
1897099118,2cyclictest14961-21expr21:10:172
1897099118,2cyclictest0-21swapper/222:45:162
1897099117,3cyclictest23287-21ssh23:12:562
1897099117,1cyclictest28516-21cut22:30:122
1897099115,2cyclictest224-21systemd-journal20:37:142
1897099110,10cyclictest3630-21sh19:40:182
1896699118,2cyclictest8893-21grep22:15:221
1896699118,2cyclictest7131-21sed19:45:341
1896699118,2cyclictest3872-21idleruntime19:40:211
1896699118,2cyclictest3732-21tr23:45:161
1896699118,2cyclictest31964-21sh21:45:121
1896699118,2cyclictest30165-21ssh00:30:101
1896699118,2cyclictest26099-21sh22:04:461
1896699118,2cyclictest25972-21ssh23:15:161
1896699118,2cyclictest24023-21ssh00:25:101
1896699118,2cyclictest2193-21ssh22:10:241
1896699118,2cyclictest2143-21ssh23:20:321
1896699118,2cyclictest2105-21wc19:35:331
1896699118,2cyclictest20392-21ssh22:47:181
1896699118,2cyclictest1365-21grep20:40:331
1896699118,2cyclictest13122-21df20:00:151
1896699117,3cyclictest5492-21tr00:10:161
1896699117,3cyclictest3726-21grep20:45:351
1896699117,3cyclictest22831-21wc22:25:261
1896699113,4cyclictest224-21systemd-journal00:36:291
1895999119,1cyclictest313252sleep000:30:230
1895999118,2cyclictest7997-21cron00:40:000
1895999118,2cyclictest6548-21sshd22:37:150
1895999118,2cyclictest6001-21ssh22:13:220
1895999118,2cyclictest3713-21grep20:45:350
1895999118,2cyclictest30241-21ssh23:17:500
1895999118,2cyclictest21371-21ssh22:00:260
1895999118,2cyclictest19815-21ssh23:55:370
1895999118,2cyclictest18227-21sed00:20:170
1895999118,2cyclictest15934-21ssh21:33:030
1895999118,2cyclictest14631-21ssh22:43:270
1895999118,2cyclictest1394-21grep19:35:220
1895999118,2cyclictest11086-21sed21:00:350
1895999118,2cyclictest10395-21tr19:55:130
1895999117,3cyclictest8031-21tr19:50:140
1895999117,3cyclictest23869-21grep20:20:360
1895999113,4cyclictest224-21systemd-journal20:20:000
1895999112,6cyclictest12079-21gdbus19:15:000
1895999111,5cyclictest5427-21apt-get19:45:120
1895999110,10cyclictest32445-21ssh21:21:270
1897299108,2cyclictest4186-21ntp_kernel_pll_19:40:243
1897299108,1cyclictest25298-21grep21:40:153
1897299108,1cyclictest16961-21seq20:08:123
1897299108,1cyclictest14445-21ssh21:10:093
1897299108,1cyclictest14445-21ssh21:10:093
1897299108,1cyclictest13637-21cat20:00:213
1897299107,2cyclictest7154-21wget20:55:163
1897299107,2cyclictest20138-21expr20:15:163
1897299107,2cyclictest11750-21cat19:55:303
1897099108,1cyclictest9696-21chrt19:52:242
1897099108,1cyclictest8986-21chrt20:58:232
1897099108,1cyclictest22012-21expr22:25:192
1897099108,1cyclictest18870-21expr20:10:332
1897099107,2cyclictest5701-21expr19:45:152
1897099107,2cyclictest3994-21ssh22:35:212
1897099105,2cyclictest224-21systemd-journal19:30:042
1897099100,5cyclictest27746-21fschecks_count20:30:172
1895999108,1cyclictest5284-21cat20:50:230
1895999108,1cyclictest31044-21ssh00:05:030
1895999108,1cyclictest268472sleep019:22:010
1895999108,1cyclictest268472sleep019:22:010
1895999108,1cyclictest16068-21ssh22:20:240
1895999107,2cyclictest28963-21taskset20:31:470
1895999107,2cyclictest24990-21ssh23:36:330
1895999107,2cyclictest12696-21cut20:00:130
1895999107,2cyclictest12623-21grep21:05:200
189599997,1cyclictest17991-21cut20:10:190
189599997,1cyclictest15154-21egrep20:05:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional