You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-19 - 01:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Thu Jul 18, 2024 12:44:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12202993014,15cyclictest28947-21ssh07:41:353
12189992623,2cyclictest0-21swapper/009:35:530
1220299251,24cyclictest34-21ksoftirqd/309:44:413
1220299250,24cyclictest17770-21ssh09:20:373
1220299250,24cyclictest0-21swapper/311:09:593
1220299250,24cyclictest0-21swapper/309:39:273
1219899250,24cyclictest6399-21diskmemload09:11:412
1219899250,24cyclictest12858-21ssh09:16:132
1219399250,24cyclictest6399-21diskmemload10:53:541
1219399250,24cyclictest6399-21diskmemload09:15:291
1220299241,1cyclictest34-21ksoftirqd/309:11:503
1220299240,24cyclictest6399-21diskmemload10:38:443
1220299240,24cyclictest6399-21diskmemload10:30:183
12198992423,1cyclictest0-21swapper/209:42:432
12198992422,1cyclictest0-21swapper/210:06:332
12198992420,3cyclictest0-21swapper/209:27:042
1219899240,24cyclictest0-21swapper/209:23:142
1219899240,23cyclictest25075-21ssh11:03:372
12193992424,0cyclictest0-21swapper/109:23:141
12193992422,1cyclictest0-21swapper/109:31:191
1219399240,24cyclictest0-21swapper/110:57:281
1219399240,24cyclictest0-21swapper/110:38:081
1219399240,23cyclictest17423-21ssh09:50:461
1219399240,23cyclictest0-21swapper/110:26:011
1219399240,1cyclictest6399-21diskmemload11:06:181
1219399240,1cyclictest27381-21ssh11:40:331
1219399240,1cyclictest0-21swapper/111:10:421
1219399240,0cyclictest0-21swapper/109:35:461
12189992422,1cyclictest0-21swapper/011:27:000
1218999240,24cyclictest6399-21diskmemload09:57:360
1218999240,23cyclictest1144-21ssh10:06:150
1218999240,23cyclictest0-21swapper/010:20:180
1218999240,0cyclictest6399-21diskmemload11:52:020
1218999240,0cyclictest0-21swapper/010:47:540
12202992321,1cyclictest0-21swapper/312:16:433
12202992321,1cyclictest0-21swapper/311:47:123
12202992320,2cyclictest31121-21sh10:02:533
12202992320,2cyclictest28162-21rm09:30:133
12202992320,2cyclictest1163-21perf09:00:003
1220299230,2cyclictest25508-21rm09:57:253
1220299230,23cyclictest6399-21diskmemload09:28:103
1220299230,23cyclictest0-21swapper/310:26:133
1220299230,23cyclictest0-21swapper/309:18:313
1220299230,22cyclictest9542-21sh12:29:473
1220299230,22cyclictest8749-21rm11:21:203
1220299230,22cyclictest3621-21ssh11:15:453
1220299230,22cyclictest20828-21ssh11:32:493
1220299230,22cyclictest19809-21ssh10:57:443
1220299230,22cyclictest15980-21ssh10:21:213
1220299230,22cyclictest0-21swapper/312:34:313
1220299230,22cyclictest0-21swapper/311:00:033
1220299230,22cyclictest0-21swapper/309:54:023
1220299230,22cyclictest0-21swapper/307:35:013
1220299230,1cyclictest0-21swapper/310:40:123
1220299230,0cyclictest6399-21diskmemload10:05:233
1220299230,0cyclictest0-21swapper/312:00:093
1220299230,0cyclictest0-21swapper/309:45:293
12198992323,0cyclictest0-21swapper/212:16:262
12198992323,0cyclictest0-21swapper/210:23:202
12198992322,1cyclictest6399-21diskmemload09:55:372
12198992322,1cyclictest30940-21ssh10:02:372
12198992321,1cyclictest25398-21ssh11:37:312
12198992321,1cyclictest12558-21apt-get09:46:272
12198992321,1cyclictest0-21swapper/210:31:422
12198992320,2cyclictest17695-21ssh11:30:162
1219899230,2cyclictest13133-21fschecks_count10:51:292
1219899230,22cyclictest5510-21sh12:25:392
1219899230,22cyclictest32147-21/usr/sbin/munin11:11:302
1219899230,22cyclictest19405-21munin-run07:25:002
1219899230,22cyclictest10027-21sh10:47:142
1219899230,1cyclictest0-21swapper/209:35:472
1219899230,0cyclictest0-21swapper/210:17:342
1219899230,0cyclictest0-21swapper/209:34:112
12193992323,0cyclictest0-21swapper/112:27:011
12193992323,0cyclictest0-21swapper/111:55:121
12193992322,1cyclictest25374-21ssh09:57:121
12193992322,0cyclictest0-21swapper/111:03:371
12193992321,1cyclictest24475-21rm12:11:301
12193992321,1cyclictest0-21swapper/110:48:051
12193992320,2cyclictest6399-21diskmemload09:46:361
1219399230,2cyclictest10763-21rm12:31:291
1219399230,23cyclictest0-21swapper/111:37:271
1219399230,22cyclictest7208-21ssh09:10:491
1219399230,22cyclictest6631-21ssh09:40:331
1219399230,22cyclictest5454-21/usr/sbin/munin11:51:321
1219399230,22cyclictest3798-21sh12:23:041
1219399230,22cyclictest1627-21sensors_temp11:46:341
1219399230,22cyclictest11204-21/usr/sbin/munin10:16:331
1219399230,1cyclictest22053-21latency_hist11:35:001
1219399230,0cyclictest6399-21diskmemload10:10:511
1219399230,0cyclictest6399-21diskmemload09:28:231
1219399230,0cyclictest0-21swapper/112:39:591
1219399230,0cyclictest0-21swapper/110:03:011
12189992323,0cyclictest6399-21diskmemload12:12:240
12189992323,0cyclictest0-21swapper/012:28:450
12189992323,0cyclictest0-21swapper/012:04:150
12189992323,0cyclictest0-21swapper/011:06:260
12189992323,0cyclictest0-21swapper/009:46:260
12189992321,1cyclictest20899-21sh10:59:420
12189992321,1cyclictest17787-21ssh09:51:190
12189992321,1cyclictest0-21swapper/010:41:210
12189992321,1cyclictest0-21swapper/009:33:380
1218999231,1cyclictest0-21swapper/009:40:570
1218999230,2cyclictest31549-21sh10:03:330
1218999230,2cyclictest18017-21rm09:21:010
1218999230,23cyclictest6399-21diskmemload09:29:350
1218999230,23cyclictest0-21swapper/012:15:290
1218999230,23cyclictest0-21swapper/010:35:540
1218999230,23cyclictest0-21swapper/009:14:110
1218999230,22cyclictest31189-21awk08:55:000
1218999230,22cyclictest24589-21latency_hist07:35:000
1218999230,22cyclictest19257-21sendmail-msp12:40:010
1218999230,22cyclictest0-21swapper/010:25:270
1218999230,1cyclictest0-21swapper/012:31:310
1218999230,1cyclictest0-21swapper/011:47:460
1218999230,0cyclictest0-21swapper/011:40:200
1218999230,0cyclictest0-21swapper/011:16:260
12202992222,0cyclictest6399-21diskmemload12:20:243
12202992222,0cyclictest0-21swapper/311:57:473
12202992222,0cyclictest0-21swapper/310:15:413
1220299222,0cyclictest34-21ksoftirqd/312:35:213
12202992220,1cyclictest16373-21munin-run10:55:003
12202992220,1cyclictest15482-21rm11:26:443
12202992220,1cyclictest0-21swapper/311:37:003
12202992220,1cyclictest0-21swapper/308:46:263
1220299220,2cyclictest29138-21ps11:41:333
1220299220,22cyclictest0-21swapper/310:10:333
1220299220,21cyclictest6399-21diskmemload11:50:103
1220299220,21cyclictest31115-21ssh11:10:323
1220299220,21cyclictest16964-21cron07:20:003
1220299220,21cyclictest0-21swapper/308:11:273
1220299220,1cyclictest25489-21ssh12:11:373
1220299220,0cyclictest151rcu_preempt10:45:423
12198992222,0cyclictest6399-21diskmemload11:53:272
12198992222,0cyclictest0-21swapper/212:35:192
12198992222,0cyclictest0-21swapper/210:10:382
12198992221,1cyclictest6399-21diskmemload11:18:132
12198992221,1cyclictest0-21swapper/210:35:412
12198992220,1cyclictest6413-21awk09:10:002
12198992220,1cyclictest4309-21perf07:59:592
12198992220,1cyclictest21564-21apt08:35:002
12198992220,1cyclictest0-21swapper/212:05:112
12198992220,1cyclictest0-21swapper/211:47:502
12198992220,1cyclictest0-21swapper/211:21:072
12198992220,0cyclictest0-21swapper/210:57:392
1219899220,22cyclictest0-21swapper/212:04:322
1219899220,21cyclictest6399-21diskmemload12:21:272
1219899220,21cyclictest6399-21diskmemload12:11:272
1219899220,21cyclictest6399-21diskmemload11:26:462
1219899220,21cyclictest18640-21systemd-journal09:05:002
1219899220,21cyclictest12314-21apt-get08:16:252
1219899220,21cyclictest10757-21apt-get12:31:292
1219899220,21cyclictest0-21swapper/211:56:262
1219899220,21cyclictest0-21swapper/211:08:132
1219899220,21cyclictest0-21swapper/210:40:332
1219899220,1cyclictest0-21swapper/209:51:222
12193992222,0cyclictest0-21swapper/112:16:261
12193992222,0cyclictest0-21swapper/112:04:331
12193992222,0cyclictest0-21swapper/111:28:481
1219399220,22cyclictest0-21swapper/110:21:091
1219399220,21cyclictest8534-21ssh11:20:581
1219399220,21cyclictest3058-21ssh10:41:241
1219399220,21cyclictest25476-21rm10:31:201
1219399220,21cyclictest19283-21ssh12:06:291
1219399220,21cyclictest0-21swapper/109:01:261
1219399220,21cyclictest0-21swapper/108:36:261
1219399220,1cyclictest22164-21cron07:30:001
12189992221,1cyclictest22359-21rm11:35:080
12189992220,2cyclictest0-21swapper/012:20:240
12189992220,1cyclictest4292-21apt-get08:00:010
12189992220,1cyclictest23970-21turbostat08:40:000
1218999220,22cyclictest0-21swapper/010:31:170
1218999220,21cyclictest31558-21sh11:11:170
1218999220,21cyclictest20320-21ssh11:31:540
1218999220,21cyclictest0-21swapper/010:16:500
1218999220,21cyclictest0-21swapper/010:10:090
1218999220,21cyclictest0-21swapper/009:16:010
1218999220,1cyclictest8968-21ssh11:55:050
1218999220,1cyclictest0-21swapper/008:31:290
1218999220,0cyclictest0-21swapper/012:05:110
1218999220,0cyclictest0-21swapper/011:23:480
12202992121,0cyclictest0-21swapper/312:05:563
1220299210,21cyclictest0-21swapper/309:00:023
1219899210,21cyclictest0-21swapper/210:26:572
1219899210,20cyclictest6399-21diskmemload11:40:222
1219899210,20cyclictest14585-21munin-run07:15:002
1219899210,0cyclictest0-21swapper/207:25:022
1219399210,21cyclictest6399-21diskmemload11:15:141
1219399210,21cyclictest0-21swapper/108:16:251
1219399210,21cyclictest0-21swapper/108:06:241
1219399210,0cyclictest19538-21apt-get08:31:291
1219399210,0cyclictest17365-21apt-get07:21:281
1219399210,0cyclictest12542-21apt-get07:11:251
1219399210,0cyclictest0-21swapper/110:05:011
1219399210,0cyclictest0-21swapper/108:00:011
1218999210,20cyclictest27473-21apt-get07:41:260
1218999210,0cyclictest0-21swapper/008:46:260
1218999210,0cyclictest0-21swapper/008:16:250
1218999210,0cyclictest0-21swapper/007:25:010
12202992020,0cyclictest0-21swapper/307:11:243
12202992017,2cyclictest7129-21idleruntime-cro08:05:003
12198992020,0cyclictest0-21swapper/208:56:282
1219899200,0cyclictest0-21swapper/208:40:002
12193992020,0cyclictest0-21swapper/107:36:261
12193992019,1cyclictest31541-21apt-get08:56:271
12193992018,1cyclictest4292-21apt-get08:00:001
12189992020,0cyclictest29869-21apt-get07:46:250
12189992020,0cyclictest24342-21apt-get08:41:260
12189992020,0cyclictest0-21swapper/009:00:010
12189992020,0cyclictest0-21swapper/008:21:260
12189992020,0cyclictest0-21swapper/008:11:270
12189992020,0cyclictest0-21swapper/007:11:250
12189992019,0cyclictest17139-21apt-get08:26:290
12189992017,2cyclictest9538-21latency_hist08:10:000
12202991919,0cyclictest0-21swapper/308:26:303
1220299190,19cyclictest0-21swapper/308:21:273
1220299190,18cyclictest0-21swapper/307:51:253
1220299190,18cyclictest0-21swapper/307:25:013
1220299190,0cyclictest0-21swapper/309:06:273
12198991919,0cyclictest0-21swapper/207:51:252
12198991919,0cyclictest0-21swapper/207:41:262
12198991918,0cyclictest0-21swapper/207:16:252
12193991918,1cyclictest32266-21apt-get07:51:251
1219399190,15cyclictest0-21swapper/108:43:331
12202991818,0cyclictest0-21swapper/307:56:283
12202991816,1cyclictest9558-21munin-run08:10:003
12198991816,1cyclictest0-21swapper/208:44:592
12198991816,1cyclictest0-21swapper/208:00:022
12198991815,1cyclictest0-21swapper/208:51:252
12193991818,0cyclictest6399-21diskmemload09:10:001
12193991818,0cyclictest30918-21/usr/sbin/munin07:46:301
12193991818,0cyclictest0-21swapper/107:41:261
12189991818,0cyclictest0-21swapper/008:56:260
12189991818,0cyclictest0-21swapper/007:21:270
12202991717,0cyclictest0-21swapper/308:41:253
1220299170,2cyclictest14362-21cron08:20:013
12193991717,0cyclictest0-21swapper/107:16:261
12189991716,1cyclictest2255-21apt-get07:56:270
12202991616,0cyclictest21928-21apt-get08:36:263
12202991614,1cyclictest0-21swapper/308:35:003
12198991616,0cyclictest0-21swapper/207:46:242
1219899160,15cyclictest18640-21systemd-journal08:05:012
1219399160,16cyclictest0-21swapper/108:27:481
1219399160,15cyclictest0-21swapper/107:31:281
12189991616,0cyclictest0-21swapper/007:51:260
12202991513,1cyclictest0-21swapper/307:36:273
12202991510,4cyclictest13599-21kworker/3:008:50:523
12198991515,0cyclictest0-21swapper/208:26:292
12198991515,0cyclictest0-21swapper/208:11:282
1219899150,0cyclictest0-21swapper/208:45:272
12193991514,0cyclictest9914-21apt-get08:11:271
12202991414,0cyclictest0-21swapper/307:45:103
12198991414,0cyclictest0-21swapper/207:31:262
1219899140,14cyclictest0-21swapper/208:20:462
1219899140,14cyclictest0-21swapper/207:35:492
12193991414,0cyclictest0-21swapper/108:50:061
12193991414,0cyclictest0-21swapper/108:21:341
12189991414,0cyclictest0-21swapper/007:35:070
12189991413,1cyclictest14939-21apt-get07:16:260
1218999140,14cyclictest0-21swapper/009:08:310
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional