You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-19 - 23:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Fri Apr 19, 2024 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18009210992,12sleep10-21swapper/107:09:051
18011210488,11sleep30-21swapper/307:09:073
17140210478,6sleep00-21swapper/007:05:200
18010210179,17sleep20-21swapper/207:09:062
245212870,1sleep1221ktimersoftd/112:30:131
54562820,1sleep30-21swapper/309:48:063
279622760,5sleep11811199cyclictest10:11:571
1811199371,34cyclictest0-21swapper/112:00:111
1811199291,2cyclictest1790-21diskmemload09:50:091
18111992716,3cyclictest0-21swapper/108:10:131
120612250,0sleep00-21swapper/009:55:110
142112240,0sleep20-21swapper/209:22:192
1811299230,18cyclictest0-21swapper/209:37:042
1811399220,3cyclictest0-21swapper/309:16:013
1811199220,2cyclictest0-21swapper/112:05:091
115572220,0sleep30-21swapper/311:40:243
1811399210,3cyclictest31191-21missed_timers07:40:173
1811399210,3cyclictest0-21swapper/307:30:253
1811399210,12cyclictest9746-21ssh10:29:413
1811299211,18cyclictest0-21swapper/212:05:042
1811299211,15cyclictest0-21swapper/211:50:052
1811199211,2cyclictest0-21swapper/108:30:121
1811199210,19cyclictest0-21swapper/110:45:091
1811199210,19cyclictest0-21swapper/107:45:101
1811399200,3cyclictest6126-21ssh09:14:203
1811399200,3cyclictest4344-21ssh10:23:373
1811399200,3cyclictest345-21diskstats07:45:123
1811399200,3cyclictest29834-21ssh11:25:203
1811399200,3cyclictest28800-21sh09:38:283
1811399200,3cyclictest24487-21telnet08:45:123
1811399200,3cyclictest21309-21ps08:35:183
1811399200,3cyclictest14724-21ssh09:57:573
1811399200,3cyclictest0-21swapper/310:10:013
1811399200,3cyclictest0-21swapper/309:30:413
1811399200,2cyclictest17591-21ssh09:25:283
1811399200,2cyclictest0-21swapper/309:50:103
1811399200,18cyclictest0-21swapper/308:40:103
1811299203,2cyclictest0-21swapper/212:18:052
18112992017,2cyclictest7752-21kworker/u8:011:37:052
18112992017,2cyclictest31006-21kworker/u8:012:13:042
18112992017,2cyclictest18450-21kworker/u8:007:17:042
18112992017,2cyclictest17409-21kworker/u8:010:17:042
18112992017,2cyclictest17409-21kworker/u8:010:09:042
18112992017,2cyclictest16339-21kworker/u8:108:29:042
18112992017,2cyclictest16205-21kworker/u8:009:29:042
18112992017,2cyclictest13795-21kworker/u8:210:39:042
18112992017,2cyclictest13766-21kworker/u8:012:23:032
18112992017,2cyclictest1336-21kworker/u8:209:57:052
18112992017,2cyclictest11763-21kworker/u8:207:24:052
1811299200,15cyclictest0-21swapper/211:11:052
1811199200,18cyclictest0-21swapper/108:40:091
1811399190,3cyclictest7248-21sensors08:00:283
1811399190,3cyclictest20029-21perl12:25:133
1811399190,3cyclictest1790-21diskmemload11:35:173
1811399190,3cyclictest17400-21ssh10:37:013
1811399190,3cyclictest12485-21ssh10:31:333
1811399190,3cyclictest0-21swapper/312:34:483
1811399190,3cyclictest0-21swapper/311:31:003
1811399190,3cyclictest0-21swapper/310:16:043
1811399190,3cyclictest0-21swapper/310:01:363
1811399190,3cyclictest0-21swapper/308:15:173
1811399190,3cyclictest0-21swapper/307:28:213
1811399190,2cyclictest224572sleep310:42:443
1811399190,2cyclictest1790-21diskmemload12:35:213
1811399190,2cyclictest1790-21diskmemload12:35:203
1811399190,2cyclictest14036-21ssh09:22:013
1811399190,2cyclictest0-21swapper/311:52:123
1811399190,2cyclictest0-21swapper/311:16:043
1811399190,17cyclictest0-21swapper/311:47:413
18112991916,2cyclictest7752-21kworker/u8:011:41:042
18112991916,2cyclictest3534-21kworker/u8:008:13:042
18112991916,2cyclictest3388-21kworker/u8:211:05:042
18112991916,2cyclictest30472-21kworker/u8:009:07:052
18112991916,2cyclictest27859-21kworker/u8:011:24:052
18112991916,2cyclictest24490-21kworker/u8:208:53:042
18112991916,2cyclictest23205-21kworker/u8:110:13:052
18112991916,2cyclictest22235-21kworker/u8:209:32:052
18112991916,2cyclictest19921-21kworker/u8:008:34:052
18112991916,2cyclictest19496-21kworker/u8:211:49:042
18112991916,2cyclictest1336-21kworker/u8:209:44:042
18112991916,2cyclictest11763-21kworker/u8:207:14:042
1811299191,2cyclictest0-21swapper/209:11:052
1811299190,15cyclictest0-21swapper/209:15:042
1811399180,2cyclictest27480-21ssh11:58:243
1811399180,2cyclictest19062-21ssh12:24:533
1811399180,2cyclictest1790-21diskmemload10:51:393
1811399180,2cyclictest10217-21munin-node3
1811399180,2cyclictest0-21swapper/310:47:253
1811399180,2cyclictest0-21swapper/310:11:303
18112991815,2cyclictest3388-21kworker/u8:211:00:052
18112991815,2cyclictest30472-21kworker/u8:009:03:042
18112991815,2cyclictest28441-21kworker/u8:208:23:052
18112991815,2cyclictest23951-21kworker/u8:012:34:042
18112991815,2cyclictest19921-21kworker/u8:008:39:042
18112991815,2cyclictest18450-21kworker/u8:007:30:052
18112991815,2cyclictest16339-21kworker/u8:108:58:052
18112991815,2cyclictest13779-21kworker/u8:112:27:052
1811299180,2cyclictest0-21swapper/210:47:042
1811399179,7cyclictest0-21swapper/312:13:023
1811399171,2cyclictest18135-21apt-get08:30:063
18113991710,6cyclictest0-21swapper/310:59:533
1811399170,3cyclictest14083-21apt-get08:20:053
1811399170,2cyclictest30164-21ssh12:00:293
1811399170,2cyclictest26309-21ssh11:21:293
1811399170,2cyclictest18200-21ssh11:13:253
18112991714,2cyclictest3534-21kworker/u8:007:54:052
18112991714,2cyclictest28441-21kworker/u8:208:18:052
18112991714,2cyclictest28441-21kworker/u8:207:57:052
18112991714,2cyclictest28441-21kworker/u8:207:49:042
1811299170,2cyclictest0-21swapper/209:52:042
1811199171,3cyclictest0-21swapper/108:55:291
1811099173,8cyclictest5393-21ssh12:10:010
1811399169,6cyclictest0-21swapper/309:05:263
1811399162,12cyclictest0-21swapper/311:04:043
1811399160,2cyclictest0-21swapper/312:09:163
18112991613,2cyclictest28441-21kworker/u8:208:06:052
18112991613,2cyclictest28441-21kworker/u8:208:04:042
1811299160,2cyclictest0-21swapper/211:34:042
1811299160,2cyclictest0-21swapper/210:34:042
1811299160,2cyclictest0-21swapper/210:01:042
1811299160,14cyclictest0-21swapper/209:45:052
1811199162,12cyclictest4515-21sh09:46:351
1811099161,3cyclictest27026-21iostat_ios07:30:160
1811399155,8cyclictest23227-21grep07:20:203
1811399154,2cyclictest1790-21diskmemload12:17:193
1811399153,3cyclictest4120-21rsyslogd08:10:003
1811399153,3cyclictest32507-21ssh09:41:533
1811399153,3cyclictest0-21swapper/307:10:223
1811399150,2cyclictest32647-21snmpd08:54:283
18112991512,2cyclictest11763-21kworker/u8:207:25:042
1811299150,8cyclictest0-21swapper/210:26:042
1811299150,2cyclictest0-21swapper/212:01:052
1811299150,13cyclictest0-21swapper/212:35:322
1811299150,13cyclictest0-21swapper/212:35:312
1811199152,7cyclictest19336-21ssh11:50:001
18111991510,3cyclictest947-21ssh09:43:241
18111991510,3cyclictest13858-21ssh09:21:441
18111991510,3cyclictest0-21swapper/111:50:051
1811099151,3cyclictest20305-21sh09:29:450
1811399143,3cyclictest4039-21aten2.4_r3power07:55:113
1811399141,3cyclictest20580-21grep07:15:143
1811399141,3cyclictest0-21swapper/309:00:133
1811399141,3cyclictest0-21swapper/308:55:123
1811399141,3cyclictest0-21swapper/308:25:123
1811399141,3cyclictest0-21swapper/308:10:013
1811399141,3cyclictest0-21swapper/307:50:163
1811399141,3cyclictest0-21swapper/307:35:103
1811299140,5cyclictest0-21swapper/210:55:052
1811299140,2cyclictest0-21swapper/208:41:052
1811299140,12cyclictest0-21swapper/211:56:352
1811299140,12cyclictest0-21swapper/210:20:552
1811199149,3cyclictest4893-21sh11:34:231
1811199142,6cyclictest0-21swapper/107:39:591
1811199141,7cyclictest0-21swapper/108:40:011
1811199140,3cyclictest1268-21timerwakeupswit09:05:281
1811199140,12cyclictest14837-21apt-get11:10:101
1811199140,12cyclictest13766-21ssh11:43:331
1811199140,12cyclictest0-21swapper/112:16:101
1811199140,12cyclictest0-21swapper/111:35:281
1811199140,12cyclictest0-21swapper/111:05:031
1811199140,12cyclictest0-21swapper/110:02:481
1811199140,12cyclictest0-21swapper/109:29:171
1811099142,7cyclictest9972-21copy10:30:010
1811099142,7cyclictest0-21swapper/011:30:000
1811099141,11cyclictest13516-21ssh12:18:200
1811099140,3cyclictest18961-21smartctl08:30:160
1811099140,3cyclictest0-21swapper/011:30:170
1811099140,12cyclictest0-21swapper/011:20:090
1811099140,12cyclictest0-21swapper/009:50:410
18112991310,2cyclictest1790-21diskmemload11:16:522
1811299130,11cyclictest0-21swapper/211:26:562
1811199138,3cyclictest18138-21munin-run08:29:591
1811199138,3cyclictest0-21swapper/110:20:391
1811199132,7cyclictest0-21swapper/107:33:221
18111991310,2cyclictest32635-21ssh10:19:251
18111991310,2cyclictest28787-21ssh09:38:281
18111991310,2cyclictest1790-21diskmemload10:35:191
18111991310,2cyclictest0-21swapper/112:12:381
18111991310,2cyclictest0-21swapper/109:55:211
18111991310,2cyclictest0-21swapper/107:20:151
18111991310,2cyclictest0-21swapper/107:10:091
1811199130,2cyclictest1790-21diskmemload10:25:571
1811199130,2cyclictest0-21swapper/112:23:451
1811199130,2cyclictest0-21swapper/111:55:231
1811199130,2cyclictest0-21swapper/111:23:161
1811199130,2cyclictest0-21swapper/110:59:131
1811199130,2cyclictest0-21swapper/110:30:131
1811199130,12cyclictest0-21swapper/111:00:481
1811199130,12cyclictest0-21swapper/107:55:191
1811199130,11cyclictest1790-21diskmemload11:16:011
1811199130,11cyclictest1790-21diskmemload09:32:281
1811199130,11cyclictest0-21swapper/111:26:381
1811099132,6cyclictest0-21swapper/009:00:010
1811099131,8cyclictest28334-21ssh12:35:000
1811099131,6cyclictest0-21swapper/008:14:590
1811099131,10cyclictest270-21jbd2/sda1-808:55:200
1811099131,10cyclictest0-21swapper/008:15:000
18110991310,2cyclictest8544-21ssh09:15:410
18110991310,2cyclictest1790-21diskmemload10:06:140
18110991310,2cyclictest0-21swapper/011:40:120
18110991310,2cyclictest0-21swapper/010:11:030
18110991310,2cyclictest0-21swapper/010:00:230
18110991310,2cyclictest0-21swapper/009:38:320
1811099130,2cyclictest0-21swapper/010:40:160
1811099130,2cyclictest0-21swapper/010:18:540
1811099130,2cyclictest0-21swapper/007:55:130
1811099130,12cyclictest0-21swapper/011:35:250
1811099130,11cyclictest0-21swapper/010:50:060
227712120,2sleep00-21swapper/009:31:110
1811299120,11cyclictest0-21swapper/210:51:152
1811299120,11cyclictest0-21swapper/210:40:212
1811199129,2cyclictest0-21swapper/112:26:561
1811199129,2cyclictest0-21swapper/109:11:291
1811199129,2cyclictest0-21swapper/108:20:141
1811199120,2cyclictest0-21swapper/109:17:081
1811199120,2cyclictest0-21swapper/107:40:151
1811199120,11cyclictest0-21swapper/112:36:171
1811199120,11cyclictest0-21swapper/112:36:171
1811199120,11cyclictest0-21swapper/110:51:161
1811199120,11cyclictest0-21swapper/110:40:171
1811199120,11cyclictest0-21swapper/110:08:271
1811199120,11cyclictest0-21swapper/109:00:171
1811099129,2cyclictest0-21swapper/012:28:320
1811099129,2cyclictest0-21swapper/012:12:540
1811099129,2cyclictest0-21swapper/011:51:370
1811099129,2cyclictest0-21swapper/011:00:330
1811099129,2cyclictest0-21swapper/010:55:230
1811099129,2cyclictest0-21swapper/010:20:260
1811099129,2cyclictest0-21swapper/009:40:130
1811099129,2cyclictest0-21swapper/009:23:170
1811099129,2cyclictest0-21swapper/008:50:180
1811099122,7cyclictest0-21swapper/011:11:590
1811099120,6cyclictest0-21swapper/011:55:000
1811099120,2cyclictest0-21swapper/012:21:370
1811099120,2cyclictest0-21swapper/011:15:330
1811099120,2cyclictest0-21swapper/011:05:220
1811099120,2cyclictest0-21swapper/010:32:030
1811099120,2cyclictest0-21swapper/009:10:030
1811099120,2cyclictest0-21swapper/008:35:140
1811099120,2cyclictest0-21swapper/007:45:240
1811099120,11cyclictest0-21swapper/012:39:320
1811099120,11cyclictest0-21swapper/012:39:310
1811099120,11cyclictest0-21swapper/011:45:330
1811099120,11cyclictest0-21swapper/010:45:150
1811099120,11cyclictest0-21swapper/010:37:120
1811099120,11cyclictest0-21swapper/009:45:130
1811099120,11cyclictest0-21swapper/008:45:140
1811099120,11cyclictest0-21swapper/008:25:220
1811099120,11cyclictest0-21swapper/008:20:060
1811099120,11cyclictest0-21swapper/007:25:230
1811099120,10cyclictest0-21swapper/012:00:180
1811299118,2cyclictest0-21swapper/207:40:112
1811299110,9cyclictest0-21swapper/207:35:002
1811199113,5cyclictest0-21swapper/108:48:371
1811199111,6cyclictest0-21swapper/108:50:271
1811199111,6cyclictest0-21swapper/108:15:241
1811099118,2cyclictest0-21swapper/007:50:130
1811099111,8cyclictest6209-21aten2.4-expect08:00:130
1811199101,5cyclictest0-21swapper/108:00:281
1811199100,7cyclictest0-21swapper/107:18:591
1811199100,3cyclictest8520-21sort08:05:121
1811099107,2cyclictest0-21swapper/007:10:210
1811099102,6cyclictest0-21swapper/007:39:590
1811099102,5cyclictest0-21swapper/007:40:580
1811099100,7cyclictest0-21swapper/009:09:580
181129990,2cyclictest0-21swapper/208:45:012
181119996,2cyclictest0-21swapper/107:24:591
181119990,7cyclictest0-21swapper/107:50:121
181109994,3cyclictest8423-21aten2.4-expect08:05:120
181109994,3cyclictest23013-21if_err_eth008:40:140
181109994,3cyclictest22339-21munin-node0
181109983,3cyclictest20326-21aten2.4_r3power07:15:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional