You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-25 - 09:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Tue Jun 25, 2024 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
227436799754748,5cyclictest2470746-21kworker/1:1+events21:54:291
227436799751747,3cyclictest2470746-21kworker/1:1+events21:44:091
227437299750742,3cyclictest217005-21kworker/3:2+events19:35:233
227437299749745,3cyclictest217005-21kworker/3:2+events19:18:503
227436799749739,5cyclictest2470746-21kworker/1:1+events23:45:401
227437299748745,2cyclictest2383653-21kworker/3:0+events21:00:213
227436799747736,5cyclictest2470746-21kworker/1:1+events00:06:031
227437099745737,5cyclictest2371159-21kworker/2:0+events21:21:472
227436799745741,3cyclictest2369871-21kworker/1:0+events21:31:361
227437299744736,7cyclictest2463601-21kworker/3:0+events22:00:233
227436799744740,3cyclictest2470746-21kworker/1:1+events23:38:441
227436799743739,3cyclictest2470746-21kworker/1:1+events23:26:161
227436799743738,4cyclictest2470746-21kworker/1:1+events21:45:211
227436799743736,6cyclictest2470746-21kworker/1:1+events23:52:321
227437099742736,5cyclictest2371159-21kworker/2:0+events21:28:252
227436799742738,3cyclictest2470746-21kworker/1:1+events23:56:071
227436799742738,3cyclictest2470746-21kworker/1:1+events23:33:361
227437299741737,3cyclictest217005-21kworker/3:2+events19:22:103
227437299741736,4cyclictest2383653-21kworker/3:0+events21:12:223
227437099741737,3cyclictest2571948-21turbostat22:54:542
227437099741735,5cyclictest2491750-21kworker/2:1+events22:55:222
227436799741737,3cyclictest2470746-21kworker/1:1+events23:44:161
227436799741737,3cyclictest2470746-21kworker/1:1+events23:24:511
227436799741737,3cyclictest2470746-21kworker/1:1+events00:00:421
227437299740737,2cyclictest2463601-21kworker/3:0+events22:20:453
227437299740732,6cyclictest2535164-21kworker/3:2+events22:30:213
227436599740735,4cyclictest2549349-21kworker/0:0+events22:51:490
227437299739735,3cyclictest2550034-21kworker/3:1+events22:40:523
227437299739735,3cyclictest217005-21kworker/3:2+events20:17:513
227437299739732,6cyclictest217005-21kworker/3:2+events20:10:203
227437299737733,3cyclictest2383653-21kworker/3:0+events20:57:213
227437299737733,3cyclictest217005-21kworker/3:2+events20:01:243
227437299736733,2cyclictest2463601-21kworker/3:0+events21:55:203
227437299736732,3cyclictest2383653-21kworker/3:0+events21:06:583
227436599736732,3cyclictest2549349-21kworker/0:0+events22:45:210
227436799733729,3cyclictest2223659-21kworker/1:0+events19:10:441
227437099731726,4cyclictest2585527-21kworker/2:0+events23:03:132
227437299729724,4cyclictest2550034-21kworker/3:1+events23:15:233
227437299728725,2cyclictest2383653-21kworker/3:0+events21:18:463
227437299728724,3cyclictest217005-21kworker/3:2+events20:06:583
227437299726716,4cyclictest217005-21kworker/3:2+events19:25:223
227437099725720,4cyclictest2371159-21kworker/2:0+events20:33:032
227437099724719,4cyclictest3351471-21kworker/2:2+events20:27:532
227437099724719,4cyclictest3351471-21kworker/2:2+events20:27:522
227437099723718,4cyclictest2371159-21kworker/2:0+events20:37:132
227437299722718,3cyclictest2606746-21kworker/3:0+events00:24:003
227437299722718,3cyclictest2606746-21kworker/3:0+events00:23:593
227437299721718,2cyclictest2722174-21kworker/3:2+events00:34:253
227436599718713,4cyclictest2420336-21kworker/0:0+events21:37:270
227437299716711,4cyclictest2463601-21kworker/3:0+events22:18:303
227437099715710,4cyclictest3351471-21kworker/2:2+events20:22:332
227437299714708,5cyclictest2606746-21kworker/3:0+events00:19:353
227436599714708,5cyclictest2456371-21turbostat21:34:560
227437299711708,2cyclictest2535164-21kworker/3:2+events22:27:513
227437299711708,2cyclictest2535164-21kworker/3:2+events22:27:513
227437099710705,4cyclictest2434624-21turbostat21:19:562
227437299709707,1cyclictest217005-21kworker/3:2+events19:33:263
227437299709704,4cyclictest2708315-21turbostat00:29:553
227437299705702,2cyclictest2332318-21turbostat19:59:543
227437299703699,3cyclictest2383625-21turbostat20:39:553
227437299701696,4cyclictest2463601-21kworker/3:0+events22:10:213
227437299700696,2cyclictest217005-21kworker/3:2+events19:52:113
227437299698695,2cyclictest2606746-21kworker/3:0+events00:10:423
227437299697694,2cyclictest2396269-21turbostat20:49:563
227437299692686,5cyclictest2463601-21kworker/3:0+events22:05:203
227436799692687,4cyclictest2449128-21turbostat21:29:551
227437299691688,2cyclictest2274208-21turbostat19:14:563
227437299688685,2cyclictest2593540-21turbostat23:09:563
227437299688682,4cyclictest217005-21kworker/3:2+events19:47:313
227437299687682,4cyclictest2383653-21kworker/3:0+events20:50:213
227437299684679,4cyclictest2722174-21kworker/3:2+events00:37:563
227436599683678,4cyclictest2557604-21turbostat22:44:540
227437299676672,3cyclictest2550034-21kworker/3:1+events23:13:253
227437299676672,3cyclictest217005-21kworker/3:2+events20:20:223
227437299675671,3cyclictest217005-21kworker/3:2+events19:43:073
227436799671667,3cyclictest2463576-21turbostat21:39:551
227436799670664,5cyclictest2607995-21turbostat23:19:561
227437299664661,2cyclictest2550034-21kworker/3:1+events22:35:233
227437299658653,4cyclictest2679603-21turbostat00:09:553
227437299648643,4cyclictest217005-21kworker/3:2+events20:44:093
227437299639635,3cyclictest2485447-21turbostat21:54:543
227437299636631,4cyclictest2586374-21turbostat23:04:563
2274370993990,397cyclictest2678903-21taskset00:02:102
25723782800,2sleep10-21swapper/122:50:131
23107592330,2sleep1301rcuc/119:35:221
2274370992620,4cyclictest2528640-21dump-pmu-power22:20:002
2274370992521,3cyclictest0-21swapper/200:00:002
2274372992421,2cyclictest521-21in:imuxsock21:35:003
2274367992318,3cyclictest0-21swapper/123:00:011
2274365992319,3cyclictest521-21in:imuxsock00:40:000
2274365992219,2cyclictest2471489-21irqcore21:40:150
2274365992219,2cyclictest2326831-21irqcore19:50:190
2274372992118,2cyclictest2443678-21kthreadcore21:20:223
2274372992118,2cyclictest0-21swapper/323:00:013
2274370992118,2cyclictest2427350-21diskmemload22:25:002
2274370992117,3cyclictest2557583-21cron22:40:002
2274367992117,3cyclictest505-21dbus-daemon20:23:261
2274367992017,2cyclictest2690160-21kthreadcore00:10:251
2274367992017,2cyclictest2536513-21cpuspeed_turbos22:25:141
2274367992017,2cyclictest2536513-21cpuspeed_turbos22:25:131
25663412190,1chrt0-21swapper/122:45:161
2274370991916,2cyclictest511-21polkitd23:34:592
2274367991916,2cyclictest2415650-21irqcore21:00:181
2274367991916,2cyclictest2403081-21gltestperf20:50:151
2274365991916,2cyclictest2615083-21sendmail23:20:010
2274365991912,1cyclictest2665230-21turbostat23:55:000
2274367991815,2cyclictest2715519-21cron00:30:001
2274367991815,2cyclictest2551298-21irqcore22:35:191
2274367991815,2cyclictest2434621-21sh21:15:011
2274365991815,2cyclictest550-21ModemManager20:22:220
2274365991815,2cyclictest514-21systemd-logind23:28:340
2274365991814,3cyclictest2522273-21irqcore22:15:210
2274365991814,2cyclictest0-21swapper/022:25:350
2274365991814,2cyclictest0-21swapper/022:25:350
2274365991812,2cyclictest25215051systemd-run22:15:000
2274372991714,2cyclictest2643775-21sendmail23:40:003
2274372991714,2cyclictest2623063-21irqcore23:25:173
2274372991710,5cyclictest364-21systemd-journal23:59:593
227437099178,8cyclictest2571938-21cron22:50:002
2274370991714,2cyclictest504-21avahi-daemon22:05:002
2274370991714,2cyclictest2623551-21tr23:25:182
2274370991714,2cyclictest2351377-21apt-key20:10:012
2274370991712,3cyclictest0-21swapper/200:30:012
2274367991714,2cyclictest2449126-21idleruntime-cro21:25:011
2274367991714,2cyclictest2413912-21missed_timers20:55:281
2274367991714,2cyclictest0-21swapper/119:20:011
2274367991713,3cyclictest2701109-21munin-plugin-st00:20:001
2274367991713,3cyclictest0-21swapper/122:00:001
2274367991712,3cyclictest0-21swapper/119:35:011
2274365991714,2cyclictest364-21systemd-journal21:56:340
2274365991714,2cyclictest2600785-21idleruntime-cro23:10:010
2274365991714,2cyclictest0-21swapper/022:24:590
2274365991713,2cyclictest0-21swapper/021:05:000
2274365991712,4cyclictest0-21swapper/000:00:000
2274370991613,2cyclictest0-21swapper/221:40:222
2274370991612,3cyclictest2644173-21cpuspeed_turbos23:40:122
2274370991612,3cyclictest2433282-21unixbench_multi21:10:322
2274370991612,3cyclictest2301283-21sh19:30:012
2274370991612,3cyclictest0-21swapper/219:55:002
2274370991612,2cyclictest0-21swapper/223:45:202
2274370991610,4cyclictest364-21systemd-journal00:14:592
2274367991613,2cyclictest815-21cron20:40:001
2274367991613,2cyclictest521-21in:imuxsock21:19:441
2274367991612,3cyclictest2601455-21gltestperf23:10:161
2274365991614,1cyclictest2686772-21turbostat.cron00:10:000
2274365991613,2cyclictest382-21systemd-udevd19:58:340
2274365991611,3cyclictest2479519-21irqrtprio21:45:220
2274365991611,3cyclictest2289378-21sort19:20:180
2274365991610,4cyclictest364-21systemd-journal00:35:000
2274372991513,1cyclictest2478075-21apt-key21:45:003
2274372991512,2cyclictest2565127-21cpuspeed_turbos22:45:103
227437099159,1cyclictest0-21swapper/220:50:002
227437099158,5cyclictest0-21swapper/222:40:142
2274370991512,2cyclictest504-21avahi-daemon00:15:322
2274370991512,2cyclictest2721728-21grep00:30:362
2274370991512,2cyclictest2607992-21sh23:15:012
2274370991512,2cyclictest2512475-21needreboot22:05:222
2274370991512,2cyclictest2427350-21diskmemload00:10:002
2274370991512,2cyclictest2274359-21cyclictest23:15:202
2274370991512,2cyclictest0-21swapper/223:40:002
2274370991512,2cyclictest0-21swapper/222:35:012
2274370991511,3cyclictest2402800-21cpuspeed_turbos20:50:122
2274367991512,2cyclictest2704521-21kthreadcore00:20:261
2274367991512,2cyclictest2704521-21kthreadcore00:20:261
2274367991512,2cyclictest2427350-21diskmemload22:15:221
2274367991512,2cyclictest2326111-21idleruntime-cro19:49:591
2274367991512,2cyclictest2274359-21cyclictest19:25:001
2274365991512,2cyclictest364-21systemd-journal23:44:340
2274365991512,2cyclictest2398753-21kthreadcore20:45:240
2274365991511,3cyclictest2620965-21unixbench_multi23:20:300
2274365991511,3cyclictest2442275-21cpuspeed_turbos21:20:150
2274365991511,2cyclictest0-21swapper/000:15:000
227437299149,4cyclictest2470377-21sshd21:38:433
227437299143,3cyclictest0-21swapper/323:50:193
2274372991410,2cyclictest0-21swapper/320:30:133
2274370991411,2cyclictest0-21swapper/223:10:002
2274370991410,2cyclictest2536456-21cpuspeed_turbos22:25:142
2274370991410,2cyclictest2536456-21cpuspeed_turbos22:25:132
227437099140,5cyclictest23107612kthreadcore19:35:222
227436799149,4cyclictest364-21systemd-journal22:20:431
227436799148,5cyclictest0-21swapper/100:35:371
2274367991413,1cyclictest440-21hwrng22:41:351
2274367991412,1cyclictest738-21runrttasks21:05:171
2274367991411,2cyclictest2716397-21gltestperf00:30:201
2274367991411,2cyclictest2519580-21missed_timers22:10:261
227436599148,5cyclictest505-21dbus-daemon23:05:010
227436599144,9cyclictest2413424-21kthreadcore20:55:250
2274365991412,1cyclictest2636573-21turbostat23:35:000
2274365991412,1cyclictest2521076-21kworker/0:1+kacpid22:33:110
2274365991411,2cyclictest364-21systemd-journal21:19:440
2274365991411,2cyclictest2376984-21unixbench_multi20:25:300
2274365991411,2cyclictest2376984-21unixbench_multi20:25:290
2274365991411,2cyclictest2307411-21kworker/0:1+kacpid19:47:470
2274365991411,2cyclictest2268295-21kworker/0:2+kacpid19:12:090
2274365991410,3cyclictest2427955-21gltestperf21:10:140
2274365991410,3cyclictest2351083-21kworker/0:2+kacpi_notify20:15:120
2274365991410,3cyclictest2307881-21cpuspeed_turbos19:35:120
2274365991410,3cyclictest0-21swapper/020:39:590
227437299138,4cyclictest364-21systemd-journal21:29:563
227437299136,5cyclictest2485093-21sshd21:48:423
227437299136,5cyclictest2371472-21apt-get20:25:113
227437299136,5cyclictest2371472-21apt-get20:25:113
2274372991310,2cyclictest2615888-21irqcore23:20:193
2274372991310,2cyclictest2574212-21kthreadcore22:50:203
2274372991310,2cyclictest0-21swapper/323:45:003
227437299130,3cyclictest0-21swapper/323:50:003
227437099139,3cyclictest731-21sshd19:59:422
227437099138,4cyclictest2322039-21tr19:45:182
227437099137,5cyclictest2427350-21diskmemload21:55:202
227437099135,7cyclictest2492627-21idleruntime-cro21:54:592
2274370991311,1cyclictest0-21swapper/219:40:402
2274370991310,2cyclictest2427344-21cron21:10:012
2274370991310,2cyclictest2274813-21gltestperf19:10:142
227436799139,3cyclictest2371825-21gltestperf20:25:161
227436799139,3cyclictest2371825-21gltestperf20:25:151
227436799139,2cyclictest309-21jbd2/sda2-820:50:001
227436799139,2cyclictest0-21swapper/120:20:011
227436799138,3cyclictest2338546-21apt-get20:00:001
2274367991310,2cyclictest2508490-21irqrtprio22:05:161
2274367991310,2cyclictest0-21swapper/122:00:211
2274367991310,2cyclictest0-21swapper/120:35:011
227436799130,12cyclictest0-21swapper/120:05:141
227436599139,3cyclictest2601156-21cpuspeed_turbos23:10:120
227436599139,3cyclictest2420336-21kworker/0:0+kacpi_notify21:25:240
227436599139,2cyclictest2500292-21cpuspeed_turbos22:00:140
227436599137,4cyclictest2699431-21ntpq00:15:290
227436599135,7cyclictest2550941-21cut22:35:140
227436599131,3cyclictest2351250-21munin-run20:10:000
2274365991310,2cyclictest2301617-21cpuspeed_turbos19:30:120
227437299129,2cyclictest2673244-21cut00:00:213
227437099129,2cyclictest2409017-21cpuspeed_turbos20:55:162
227437099129,2cyclictest2363080-21proc_pri20:15:292
227437099129,2cyclictest0-21swapper/223:20:292
227437099125,5cyclictest530-21gdbus20:05:002
2274370991210,1cyclictest0-21swapper/223:51:562
227437099120,1cyclictest0-21swapper/222:11:432
227437099120,1cyclictest0-21swapper/200:24:402
227437099120,1cyclictest0-21swapper/200:24:402
227437099120,11cyclictest0-21swapper/221:02:042
227436799129,2cyclictest0-21swapper/119:25:331
227436799127,4cyclictest2543930-21gltestperf22:30:171
227436799127,4cyclictest0-21swapper/123:00:291
227436599129,2cyclictest2426959-21unixbench_singl21:05:280
227436599129,2cyclictest2383347-21kworker/0:2+kacpid20:42:520
227436599129,2cyclictest2307411-21kworker/0:1+kacpid19:43:110
227436599129,2cyclictest2301284-21munin-run19:30:010
227436599128,3cyclictest2420336-21kworker/0:0+kacpid21:51:110
227436599128,2cyclictest0-21swapper/023:45:120
227436599126,4cyclictest2377708-21apt-get20:30:110
2274365991210,1cyclictest2700045-21kworker/0:2+kacpid00:25:040
2274365991210,1cyclictest2665063-21kworker/0:1+kacpid00:03:510
2274365991210,1cyclictest2593516-21kworker/0:1+kacpid23:39:480
227436599120,5cyclictest2507891-21sed22:05:160
22741752122,2sleep00-21swapper/019:09:490
227437299117,2cyclictest0-21swapper/323:30:153
227437099119,1cyclictest0-21swapper/219:25:012
227437099118,2cyclictest0-21swapper/221:45:352
227437099118,2cyclictest0-21swapper/221:35:292
227437099117,3cyclictest2463604-21systemd-run21:35:002
227437099117,2cyclictest0-21swapper/220:15:002
227437099110,11cyclictest0-21swapper/220:42:122
227436799119,1cyclictest0-21swapper/120:10:191
227436799119,1cyclictest0-21swapper/119:52:441
227436799118,2cyclictest2593935-21cpuspeed_turbos23:05:121
227436799116,4cyclictest2340199-21sed20:00:181
227436799110,1cyclictest0-21swapper/119:43:481
227436599119,1cyclictest2268295-21kworker/0:2+kacpid19:16:500
227436599118,2cyclictest0-21swapper/020:15:000
227436599118,2cyclictest0-21swapper/020:00:110
227436599117,3cyclictest2403009-21gltestperf20:50:150
227436599117,2cyclictest2701611-21df_inode00:20:180
227436599117,2cyclictest2701611-21df_inode00:20:170
227436599110,8cyclictest2586549-21cat23:00:010
22740232118,2sleep30-21swapper/319:07:503
227437099108,1cyclictest0-21swapper/219:15:032
227437099105,4cyclictest2307258-21seq19:31:522
227437099100,4cyclictest0-21swapper/200:36:102
227436799107,2cyclictest0-21swapper/120:40:321
22735112102,4sleep10-21swapper/119:05:331
2273930296,2sleep20-21swapper/219:06:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional