You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-10-01 - 00:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Sep 30, 2023 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5162992623,2cyclictest23239-21apt08:50:573
515099260,2cyclictest30214-21date10:15:010
5150992523,1cyclictest0-21swapper/011:50:510
5150992523,1cyclictest0-21swapper/009:27:170
5162992420,3cyclictest6935-21perf11:30:003
516299240,23cyclictest32559-21diskmemload10:45:313
516299240,23cyclictest32559-21diskmemload10:03:203
516299240,23cyclictest0-21swapper/310:38:133
516299240,1cyclictest0-21swapper/309:48:203
5157992423,1cyclictest32559-21diskmemload11:29:392
5157992422,1cyclictest2825-21ssh09:11:132
5157992422,1cyclictest2825-21ssh09:11:132
515799240,24cyclictest32559-21diskmemload11:35:182
515799240,23cyclictest32559-21diskmemload12:01:462
515799240,23cyclictest32559-21diskmemload11:55:362
515799240,23cyclictest29289-21ssh10:46:062
515799240,23cyclictest11577-21rm10:28:462
515799240,23cyclictest0-21swapper/209:39:322
5153992424,0cyclictest32559-21diskmemload09:17:111
5153992423,1cyclictest32559-21diskmemload10:47:171
515399240,23cyclictest23875-21ssh11:14:111
515399240,23cyclictest0-21swapper/109:51:421
515399240,1cyclictest32559-21diskmemload10:08:451
515399240,0cyclictest0-21swapper/112:09:561
5150992422,1cyclictest16631-21cron11:40:000
5150992422,1cyclictest0-21swapper/012:16:520
515099240,23cyclictest32559-21diskmemload10:28:530
515099240,23cyclictest20445-21apt-get08:45:550
515099240,23cyclictest0-21swapper/012:36:280
515099240,23cyclictest0-21swapper/010:18:590
515099240,23cyclictest0-21swapper/009:11:070
515099240,23cyclictest0-21swapper/009:11:070
515099240,0cyclictest0-21swapper/012:09:390
5162992323,0cyclictest32559-21diskmemload12:14:013
5162992323,0cyclictest0-21swapper/311:51:033
5162992323,0cyclictest0-21swapper/311:11:263
5162992322,1cyclictest34-21ksoftirqd/311:07:473
5162992321,1cyclictest0-21swapper/309:40:573
5162992321,1cyclictest0-21swapper/309:35:073
516299231,22cyclictest331rcuc/309:15:253
516299230,2cyclictest11026-21/usr/sbin/munin09:21:003
516299230,23cyclictest0-21swapper/311:24:283
516299230,23cyclictest0-21swapper/311:04:063
516299230,23cyclictest0-21swapper/310:15:193
516299230,22cyclictest8890-21latency_hist12:05:003
516299230,22cyclictest16015-21sh11:38:583
516299230,22cyclictest0-21swapper/312:37:553
516299230,22cyclictest0-21swapper/312:15:023
516299230,22cyclictest0-21swapper/311:59:323
516299230,22cyclictest0-21swapper/309:51:303
516299230,0cyclictest32559-21diskmemload10:28:253
516299230,0cyclictest0-21swapper/311:40:163
516299230,0cyclictest0-21swapper/309:58:163
516299230,0cyclictest0-21swapper/309:27:433
516299230,0cyclictest0-21swapper/309:11:303
516299230,0cyclictest0-21swapper/309:11:303
5157992323,0cyclictest32559-21diskmemload12:31:542
5157992323,0cyclictest32559-21diskmemload09:42:442
5157992323,0cyclictest0-21swapper/209:20:302
5157992322,1cyclictest20326-21ssh10:04:552
5157992321,1cyclictest0-21swapper/209:25:582
515799231,21cyclictest19948-21gzip07:40:002
515799231,21cyclictest19948-21gzip07:40:002
515799230,2cyclictest19158-21cat11:10:002
515799230,23cyclictest0-21swapper/212:09:042
515799230,23cyclictest0-21swapper/211:51:062
515799230,23cyclictest0-21swapper/210:54:502
515799230,22cyclictest8375-21sh09:17:372
515799230,22cyclictest4275-21apt-get10:55:022
515799230,22cyclictest0-21swapper/209:46:452
515799230,1cyclictest32559-21diskmemload11:11:282
515799230,0cyclictest32559-21diskmemload10:41:032
515799230,0cyclictest32559-21diskmemload10:39:042
515799230,0cyclictest32559-21diskmemload09:57:082
515799230,0cyclictest0-21swapper/212:36:302
515799230,0cyclictest0-21swapper/211:01:272
515799230,0cyclictest0-21swapper/209:51:062
515799230,0cyclictest0-21swapper/209:31:242
5153992323,0cyclictest0-21swapper/112:18:181
5153992323,0cyclictest0-21swapper/111:51:121
5153992323,0cyclictest0-21swapper/110:57:081
5153992323,0cyclictest0-21swapper/109:57:261
5153992323,0cyclictest0-21swapper/109:35:031
5153992323,0cyclictest0-21swapper/109:31:191
5153992323,0cyclictest0-21swapper/109:20:341
5153992322,0cyclictest0-21swapper/111:56:081
5153992321,1cyclictest19353-21mailstats10:36:051
5153992320,2cyclictest7926-21ssh10:25:241
5153992320,2cyclictest30156-21ssh11:20:271
5153992320,2cyclictest17078-21ssh09:26:191
515399230,22cyclictest9900-21ssh12:38:221
515399230,22cyclictest7727-21munin-run08:20:001
515399230,22cyclictest26822-21avahi-daemon09:00:001
515399230,1cyclictest16086-21ssh10:00:321
515399230,1cyclictest10010-21ssh11:00:281
515399230,1cyclictest0-21swapper/111:36:081
515399230,0cyclictest0-21swapper/112:21:091
515399230,0cyclictest0-21swapper/112:12:381
515399230,0cyclictest0-21swapper/111:30:511
515399230,0cyclictest0-21swapper/110:42:451
5150992323,0cyclictest32559-21diskmemload11:55:220
5150992323,0cyclictest32559-21diskmemload10:45:200
5150992323,0cyclictest0-21swapper/012:11:030
5150992323,0cyclictest0-21swapper/011:26:290
5150992323,0cyclictest0-21swapper/011:10:130
5150992323,0cyclictest0-21swapper/011:02:450
5150992323,0cyclictest0-21swapper/010:07:240
5150992322,1cyclictest0-21swapper/010:01:060
5150992321,1cyclictest0-21swapper/010:38:380
5150992320,2cyclictest7710-21perf08:20:000
515099231,21cyclictest597-21dbus-daemon10:00:000
515099230,2cyclictest18640-21systemd-journal07:15:000
515099230,23cyclictest0-21swapper/011:42:360
515099230,23cyclictest0-21swapper/011:20:360
515099230,23cyclictest0-21swapper/010:52:260
515099230,23cyclictest0-21swapper/010:43:000
515099230,22cyclictest26378-21sh11:16:020
515099230,22cyclictest0-21swapper/009:16:420
515099230,21cyclictest0-21swapper/009:20:470
515099230,0cyclictest0-21swapper/012:34:090
515099230,0cyclictest0-21swapper/012:26:430
515099230,0cyclictest0-21swapper/010:55:560
515099230,0cyclictest0-21swapper/010:20:520
515099230,0cyclictest0-21swapper/009:38:580
515099230,0cyclictest0-21swapper/009:31:590
5162992222,0cyclictest34-21ksoftirqd/309:30:573
5162992222,0cyclictest0-21swapper/312:31:463
5162992222,0cyclictest0-21swapper/311:45:373
5162992221,0cyclictest0-21swapper/310:20:163
5162992220,1cyclictest17845-21apt-get07:35:533
5162992220,1cyclictest17845-21apt-get07:35:533
5162992220,1cyclictest10677-21memory12:06:083
5162992220,1cyclictest0-21swapper/311:30:503
516299220,22cyclictest0-21swapper/311:15:253
516299220,21cyclictest32559-21diskmemload10:55:513
516299220,21cyclictest32559-21diskmemload10:30:303
516299220,21cyclictest0-21swapper/310:42:333
516299220,21cyclictest0-21swapper/310:10:563
516299220,21cyclictest0-21swapper/307:55:583
516299220,21cyclictest0-21swapper/307:45:023
516299220,21cyclictest0-21swapper/307:10:563
516299220,1cyclictest9937-21idleruntime-cro07:20:003
516299220,1cyclictest0-21swapper/310:52:513
516299220,0cyclictest0-21swapper/312:28:153
5157992222,0cyclictest0-21swapper/210:32:212
5157992220,1cyclictest9946-21sendmail-msp07:20:002
5157992220,1cyclictest9669-21ssh11:31:232
5157992220,1cyclictest20936-21rm12:16:112
5157992220,1cyclictest18640-21systemd-journal08:00:002
515799220,22cyclictest0-21swapper/211:46:422
515799220,22cyclictest0-21swapper/211:24:232
515799220,21cyclictest31221-21timerwakeupswit12:26:112
515799220,21cyclictest15154-21ssh12:10:582
515799220,21cyclictest0-21swapper/212:20:262
515799220,21cyclictest0-21swapper/211:40:022
515799220,21cyclictest0-21swapper/210:11:022
515799220,1cyclictest32559-21diskmemload11:15:552
515799220,1cyclictest324-21ssh10:16:112
515799220,1cyclictest17709-21turbostat08:40:012
515799220,0cyclictest0-21swapper/210:20:092
5153992222,0cyclictest0-21swapper/111:06:031
5153992222,0cyclictest0-21swapper/110:20:261
5153992222,0cyclictest0-21swapper/110:12:531
5153992222,0cyclictest0-21swapper/108:30:531
5153992220,1cyclictest605-21rs:main1
515399220,22cyclictest26184-21ssh11:49:031
515399220,22cyclictest15144-21sh10:31:301
515399220,22cyclictest0-21swapper/112:34:511
515399220,22cyclictest0-21swapper/111:25:071
515399220,21cyclictest4510-21ssh12:00:251
515399220,21cyclictest28077-21sh11:17:211
515399220,21cyclictest25862-21apt-get07:50:021
515399220,21cyclictest1329-21ssh09:45:051
515399220,1cyclictest0-21swapper/111:42:211
515399220,0cyclictest0-21swapper/110:50:041
515399220,0cyclictest0-21swapper/109:10:411
515399220,0cyclictest0-21swapper/109:10:411
5150992222,0cyclictest32559-21diskmemload10:31:320
5150992222,0cyclictest0-21swapper/011:31:200
5150992222,0cyclictest0-21swapper/008:20:570
5150992220,1cyclictest28927-21sh09:40:050
515099220,21cyclictest32559-21diskmemload09:45:130
515099220,21cyclictest2913-21munin-run08:10:000
515099220,21cyclictest0-21swapper/012:00:370
515099220,21cyclictest0-21swapper/011:45:120
515099220,1cyclictest25260-21munin-plugin-st07:50:010
515099220,1cyclictest23134-21apt-get08:50:010
515099220,0cyclictest0-21swapper/011:05:070
515099220,0cyclictest0-21swapper/009:50:410
5162992119,1cyclictest7676-21sendmail08:20:003
5162992119,1cyclictest597-21dbus-daemon09:05:003
516299210,0cyclictest10839-21apt-get08:25:573
516299210,0cyclictest0-21swapper/312:20:123
515799210,21cyclictest0-21swapper/210:06:242
515799210,1cyclictest5450-21apt-get07:10:552
515799210,1cyclictest0-21swapper/208:50:012
5153992121,0cyclictest0-21swapper/109:40:011
5153992121,0cyclictest0-21swapper/108:25:021
5153992120,1cyclictest22505-21apt-get08:50:011
5153992120,0cyclictest8049-21apt-get08:20:571
515399210,21cyclictest0-21swapper/112:25:101
515399210,21cyclictest0-21swapper/110:15:091
515399210,20cyclictest0-21swapper/107:25:021
515399210,0cyclictest0-21swapper/107:10:561
5150992121,0cyclictest0-21swapper/008:30:530
5150992121,0cyclictest0-21swapper/008:10:550
5150992119,1cyclictest27713-21sendmail09:00:000
515099210,21cyclictest0-21swapper/012:20:040
515099210,0cyclictest18049-21apt-get08:40:550
515099210,0cyclictest0-21swapper/007:30:560
5162992020,0cyclictest0-21swapper/308:40:553
5162992020,0cyclictest0-21swapper/307:30:563
5162992019,1cyclictest0-21swapper/308:30:533
516299200,20cyclictest0-21swapper/307:50:023
516299200,19cyclictest0-21swapper/307:25:023
5157992020,0cyclictest28058-21apt-get09:00:552
515799200,19cyclictest22461-21dump-pmu-power07:45:002
515799200,0cyclictest25964-21apt-get07:50:552
5153992020,0cyclictest0-21swapper/108:35:571
5150992020,0cyclictest7843-21apt-get07:15:540
5150992020,0cyclictest0-21swapper/007:55:580
5150992019,0cyclictest0-21swapper/007:50:020
515099200,0cyclictest0-21swapper/009:05:560
5162991919,0cyclictest0-21swapper/308:35:573
5157991919,0cyclictest0-21swapper/209:05:562
5157991919,0cyclictest0-21swapper/208:30:542
5153991919,0cyclictest0-21swapper/109:00:551
5150991917,1cyclictest12347-21munin-plugin-st07:25:000
515099190,0cyclictest0-21swapper/007:40:560
5162991817,0cyclictest0-21swapper/308:20:573
5157991818,0cyclictest0-21swapper/208:40:552
5157991817,1cyclictest10117-21apt-get08:25:022
5157991816,1cyclictest18640-21systemd-journal07:35:002
5153991818,0cyclictest30844-21apt-get08:00:561
5153991818,0cyclictest0-21swapper/109:05:561
5153991817,1cyclictest20388-21apt-get07:40:561
5150991818,0cyclictest0-21swapper/007:25:020
5150991816,1cyclictest0-21swapper/009:00:550
5162991717,0cyclictest0-21swapper/308:15:003
5162991716,1cyclictest0-21swapper/308:55:563
5153991716,0cyclictest0-21swapper/107:30:561
515399170,16cyclictest18899-21uname07:36:011
515399170,16cyclictest18899-21uname07:36:011
515099170,16cyclictest0-21swapper/008:35:570
5162991615,0cyclictest0-21swapper/308:05:553
5162991615,0cyclictest0-21swapper/308:00:003
5153991614,1cyclictest0-21swapper/108:50:001
5153991614,1cyclictest0-21swapper/108:45:001
515399160,15cyclictest4943-21users08:11:061
516299156,9cyclictest331rcuc/307:41:033
5162991515,0cyclictest0-21swapper/309:05:563
516299151,4cyclictest0-21swapper/307:25:003
5157991515,0cyclictest0-21swapper/208:55:552
5157991515,0cyclictest0-21swapper/207:45:522
5157991515,0cyclictest0-21swapper/207:25:542
5157991514,1cyclictest465-21munin-run08:05:002
5153991515,0cyclictest0-21swapper/107:55:581
5153991514,1cyclictest0-21swapper/107:20:591
515399150,1cyclictest1876-21ntp_states08:06:011
5150991515,0cyclictest0-21swapper/008:00:560
516299140,14cyclictest0-21swapper/308:45:413
515799140,14cyclictest0-21swapper/208:47:502
515799140,14cyclictest0-21swapper/208:15:402
515799140,14cyclictest0-21swapper/208:05:202
515799140,14cyclictest0-21swapper/207:20:082
515799140,0cyclictest0-21swapper/208:20:042
515799140,0cyclictest0-21swapper/208:10:292
5153991414,0cyclictest0-21swapper/107:45:211
515099140,14cyclictest0-21swapper/007:35:060
515099140,14cyclictest0-21swapper/007:35:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional