You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-23 - 20:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Mon Jun 23, 2025 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
230992123112,6sleep30-21swapper/307:06:273
23246212096,19sleep00-21swapper/007:08:200
22248211389,16sleep10-21swapper/107:05:161
23205210887,16sleep20-21swapper/207:07:502
274352750,0sleep2301ktimersoftd/210:02:162
259162740,1sleep00-21swapper/012:10:530
98722720,3sleep229-21rcuc/209:45:042
73942670,0sleep20-21swapper/210:47:322
23404994829,17cyclictest0-21swapper/208:50:142
165272280,0sleep00-21swapper/009:50:350
2340399221,19cyclictest7443-21diskmemload12:00:131
293602210,1sleep00-21swapper/009:30:550
2340399211,2cyclictest0-21swapper/112:20:131
2340199210,19cyclictest0-21swapper/009:45:180
23405992017,2cyclictest32673-21kworker/u8:110:18:053
24072190,0sleep10-21swapper/111:48:051
23405991916,2cyclictest19062-21kworker/u8:112:11:033
2340499190,2cyclictest0-21swapper/209:15:132
23401991916,2cyclictest2264-21kworker/u8:111:48:040
2340199191,16cyclictest7443-21diskmemload10:40:140
23405991815,2cyclictest31096-21kworker/u8:111:18:053
23405991814,2cyclictest22054-21kworker/u8:010:36:043
23405991813,3cyclictest28887-21kworker/u8:312:19:053
23405991714,2cyclictest8442-21kworker/u8:112:26:013
2340599170,15cyclictest0-21swapper/310:45:393
23401991713,2cyclictest28772-21sh10:36:030
23405991613,2cyclictest8456-21kworker/u8:011:34:053
23405991613,2cyclictest1605-21kworker/u8:109:17:043
23405991613,2cyclictest15145-21kworker/u8:210:32:043
23405991613,2cyclictest15145-21kworker/u8:210:26:053
23405991613,2cyclictest12699-21kworker/u8:009:38:013
23405991612,2cyclictest19301-21ssh11:00:163
23405991611,3cyclictest29215-21ssh10:04:593
2340499161,3cyclictest4841-21ssh10:12:162
2340399161,13cyclictest27660-21apt-get07:20:141
2340199161,3cyclictest9164-21chrt11:55:130
23405991512,2cyclictest28887-21kworker/u8:312:20:043
23405991512,2cyclictest27178-21kworker/u8:207:32:043
23405991512,2cyclictest26190-21ssh11:06:043
23405991512,2cyclictest13707-21kworker/u8:211:52:043
23405991512,2cyclictest12699-21kworker/u8:009:29:043
2340599151,12cyclictest2672-21perl10:10:203
23405991510,3cyclictest0-21swapper/312:37:293
23405991510,3cyclictest0-21swapper/310:40:283
2340599150,13cyclictest0-21swapper/311:11:333
23404991512,2cyclictest0-21swapper/212:20:162
2340499151,12cyclictest17794-21apt-get08:15:172
23404991510,3cyclictest7443-21diskmemload11:10:202
23404991510,3cyclictest0-21swapper/211:22:492
2340399151,3cyclictest18369-21iostat_ios09:20:181
2340399151,12cyclictest20593-21memory08:20:181
23403991510,3cyclictest10840-21mailstats07:55:281
2340399150,3cyclictest0-21swapper/112:17:461
2340399150,3cyclictest0-21swapper/110:27:481
2340399150,13cyclictest7443-21diskmemload12:31:301
2340399150,13cyclictest7443-21diskmemload11:25:051
2340399150,13cyclictest0-21swapper/110:45:351
2340399150,13cyclictest0-21swapper/110:22:501
2340199151,3cyclictest0-21swapper/010:25:180
23401991512,2cyclictest8456-21kworker/u8:011:34:010
23401991512,2cyclictest8456-21kworker/u8:011:24:040
23401991512,2cyclictest28887-21kworker/u8:312:35:050
23401991512,2cyclictest15145-21kworker/u8:210:31:040
2340199151,12cyclictest0-21swapper/010:55:180
2340199151,12cyclictest0-21swapper/009:27:120
23401991510,3cyclictest1441-21munin-node0
2340199150,2cyclictest16100-21ssh09:18:140
23405991411,2cyclictest4995-21kworker/u8:209:52:043
23405991411,2cyclictest2264-21kworker/u8:111:55:043
23405991411,2cyclictest21532-21kworker/u8:008:47:043
23405991411,2cyclictest12699-21kworker/u8:009:21:043
2340599140,2cyclictest0-21swapper/312:34:053
2340599140,13cyclictest0-21swapper/309:57:083
2340599140,13cyclictest0-21swapper/309:14:293
2340599140,13cyclictest0-21swapper/308:55:193
2340599140,13cyclictest0-21swapper/307:55:293
2340599140,12cyclictest4790-21ssh09:40:033
2340599140,12cyclictest22105-21fschecks_count11:35:173
2340599140,12cyclictest14451-21munin-node3
2340599140,12cyclictest0-21swapper/311:40:113
2340499149,3cyclictest7443-21diskmemload11:18:392
2340499149,3cyclictest0-21swapper/209:42:282
23404991411,2cyclictest0-21swapper/211:49:462
23404991411,2cyclictest0-21swapper/211:27:162
2340499140,13cyclictest0-21swapper/211:00:162
2340499140,12cyclictest5129-21runrttasks10:40:082
2340499140,12cyclictest28708-21ssh11:41:162
2340499140,12cyclictest0-21swapper/212:08:262
2340399149,3cyclictest6855-21sh11:52:041
2340399142,6cyclictest0-21swapper/112:05:001
2340399142,6cyclictest0-21swapper/107:15:011
2340399141,3cyclictest7241-21sh09:41:231
23403991411,2cyclictest0-21swapper/110:42:501
23403991410,2cyclictest1021-21ps09:35:241
2340399140,3cyclictest0-21swapper/111:17:511
2340399140,13cyclictest0-21swapper/111:09:201
2340399140,13cyclictest0-21swapper/111:00:201
2340399140,13cyclictest0-21swapper/109:20:001
2340399140,12cyclictest8342-21sed11:20:281
2340399140,12cyclictest0-21swapper/112:36:061
2340399140,12cyclictest0-21swapper/112:26:391
2340199149,3cyclictest8433-21ssh09:43:110
2340199149,3cyclictest7443-21diskmemload10:45:350
2340199149,3cyclictest10460-21ssh09:12:070
2340199149,3cyclictest0-21swapper/012:15:250
2340199142,6cyclictest9181-21ssh10:50:010
23401991411,2cyclictest8456-21kworker/u8:011:25:030
23401991411,2cyclictest32673-21kworker/u8:110:16:040
23401991411,2cyclictest13707-21kworker/u8:211:43:040
23401991411,2cyclictest12699-21kworker/u8:009:20:040
23401991411,2cyclictest0-21swapper/012:09:230
23401991411,2cyclictest0-21swapper/011:53:220
23401991411,2cyclictest0-21swapper/011:36:230
2340199140,2cyclictest0-21swapper/012:34:050
2340199140,2cyclictest0-21swapper/011:04:020
2340199140,12cyclictest0-21swapper/008:35:120
2340199140,10cyclictest0-21swapper/009:57:040
23405991310,2cyclictest4995-21kworker/u8:209:47:043
23405991310,2cyclictest31308-21ssh10:05:413
23405991310,2cyclictest29879-21kworker/u8:208:52:013
23405991310,2cyclictest23047-21kworker/u8:107:16:003
23405991310,2cyclictest21532-21kworker/u8:009:06:043
23405991310,2cyclictest0-21swapper/312:05:193
23405991310,2cyclictest0-21swapper/311:46:203
23405991310,2cyclictest0-21swapper/310:52:103
23405991310,2cyclictest0-21swapper/308:35:133
23405991310,2cyclictest0-21swapper/307:35:183
2340599130,2cyclictest0-21swapper/312:02:503
2340599130,2cyclictest0-21swapper/311:25:343
2340599130,2cyclictest0-21swapper/309:30:303
2340599130,12cyclictest0-21swapper/311:23:383
2340599130,12cyclictest0-21swapper/310:20:013
2340599130,11cyclictest3518-21cpu07:40:163
2340599130,11cyclictest16618-21ssh10:56:333
2340499138,3cyclictest0-21swapper/211:35:282
23404991310,2cyclictest0-21swapper/210:32:162
23404991310,2cyclictest0-21swapper/208:25:162
2340499130,3cyclictest2490-21ssh09:36:592
2340499130,2cyclictest0-21swapper/210:50:262
2340499130,2cyclictest0-21swapper/209:20:572
2340499130,2cyclictest0-21swapper/207:50:192
2340499130,12cyclictest30569-21ssh10:38:462
2340499130,12cyclictest0-21swapper/211:57:362
2340499130,12cyclictest0-21swapper/209:55:182
2340499130,12cyclictest0-21swapper/209:05:192
2340399138,3cyclictest0-21swapper/108:40:001
2340399132,6cyclictest0-21swapper/109:10:001
2340399132,6cyclictest0-21swapper/108:20:001
23403991310,2cyclictest0-21swapper/111:57:051
23403991310,2cyclictest0-21swapper/111:12:351
23403991310,2cyclictest0-21swapper/110:59:141
23403991310,2cyclictest0-21swapper/109:59:481
23403991310,2cyclictest0-21swapper/109:13:381
23403991310,2cyclictest0-21swapper/107:50:261
2340399130,2cyclictest0-21swapper/110:18:501
2340399130,2cyclictest0-21swapper/109:27:561
2340399130,2cyclictest0-21swapper/107:44:081
2340399130,1cyclictest0-21swapper/111:35:291
2340399130,12cyclictest0-21swapper/110:39:571
2340399130,12cyclictest0-21swapper/110:31:101
2340399130,11cyclictest0-21swapper/110:05:451
23401991310,2cyclictest24278-21kworker/u8:011:06:040
23401991310,2cyclictest12699-21kworker/u8:010:07:040
23401991310,2cyclictest11741-21copy10:20:030
2340199130,3cyclictest4385-21ssh10:11:360
2340199130,2cyclictest0-21swapper/007:35:270
2340199130,12cyclictest7443-21diskmemload11:10:330
2340199130,12cyclictest0-21swapper/007:44:140
2340599129,2cyclictest21532-21kworker/u8:008:28:043
2340599129,2cyclictest15942-21kworker/u8:108:22:043
2340599122,6cyclictest0-21swapper/307:45:243
2340599120,2cyclictest0-21swapper/308:40:223
2340499129,2cyclictest9773-21ssh10:17:062
2340499129,2cyclictest14776-21hddtemp_smartct12:00:182
2340499129,2cyclictest0-21swapper/211:08:472
2340499129,2cyclictest0-21swapper/210:22:372
2340499129,2cyclictest0-21swapper/209:54:292
2340499129,2cyclictest0-21swapper/209:12:392
2340499122,7cyclictest0-21swapper/212:32:592
2340499122,6cyclictest0-21swapper/209:30:302
2340499122,6cyclictest0-21swapper/208:25:002
2340499122,5cyclictest0-21swapper/211:35:002
2340499120,2cyclictest7443-21diskmemload10:08:082
2340499120,2cyclictest0-21swapper/212:10:262
2340499120,2cyclictest0-21swapper/208:10:192
2340499120,11cyclictest0-21swapper/212:35:292
2340499120,11cyclictest0-21swapper/211:50:172
2340499120,11cyclictest0-21swapper/210:26:462
2340499120,11cyclictest0-21swapper/208:05:222
2340399129,2cyclictest0-21swapper/111:30:271
2340399129,2cyclictest0-21swapper/110:50:161
2340399129,2cyclictest0-21swapper/109:51:081
2340399129,2cyclictest0-21swapper/108:50:271
2340399129,2cyclictest0-21swapper/108:25:171
2340399127,3cyclictest25165-21fschecks_count10:00:171
2340399127,3cyclictest24348-21tr12:10:171
2340399127,3cyclictest0-21swapper/107:45:181
2340399121,9cyclictest3474-21munin-node1
2340399121,9cyclictest29072-21grep09:30:321
2340399121,9cyclictest0-21swapper/108:30:181
2340399120,2cyclictest0-21swapper/111:40:141
2340399120,2cyclictest0-21swapper/109:45:071
2340399120,11cyclictest0-21swapper/110:11:151
2340399120,10cyclictest0-21swapper/108:10:221
2340199129,2cyclictest21532-21kworker/u8:008:33:040
2340199129,2cyclictest21532-21kworker/u8:008:26:040
2340199129,2cyclictest0-21swapper/011:19:220
2340199129,2cyclictest0-21swapper/008:45:190
2340199129,2cyclictest0-21swapper/008:10:150
2340199129,2cyclictest0-21swapper/007:10:150
2340199120,2cyclictest0-21swapper/008:15:180
2340199120,2cyclictest0-21swapper/007:15:130
2340199120,11cyclictest7443-21diskmemload12:21:450
2340199120,11cyclictest7443-21diskmemload10:01:020
2340199120,11cyclictest0-21swapper/012:00:380
2340199120,11cyclictest0-21swapper/009:35:080
2340599118,2cyclictest29822-21kworker/u8:107:54:013
2340599118,2cyclictest27178-21kworker/u8:207:22:053
2340599118,2cyclictest15942-21kworker/u8:108:11:043
2340599118,2cyclictest11179-21kworker/u8:008:16:053
2340499118,2cyclictest0-21swapper/212:19:562
2340499112,5cyclictest0-21swapper/208:50:002
2340499112,5cyclictest0-21swapper/208:45:002
2340499112,5cyclictest0-21swapper/208:34:592
2340499112,5cyclictest0-21swapper/208:00:012
2340499111,6cyclictest0-21swapper/207:15:242
2340499111,5cyclictest0-21swapper/207:15:012
2340499110,6cyclictest0-21swapper/208:59:592
2340499110,6cyclictest0-21swapper/207:40:012
2340499110,10cyclictest0-21swapper/212:29:062
2340499110,10cyclictest0-21swapper/209:27:562
2340399111,8cyclictest0-21swapper/107:25:141
2340399111,3cyclictest1441-21munin-node1
2340399111,3cyclictest1317-21munin-node1
2340399110,9cyclictest0-21swapper/108:40:261
2340199118,2cyclictest29879-21kworker/u8:208:54:040
2340199118,2cyclictest29822-21kworker/u8:107:28:050
2340199118,2cyclictest27178-21kworker/u8:207:34:040
2340199118,2cyclictest21532-21kworker/u8:009:01:040
2340199118,2cyclictest0-21swapper/007:45:240
2340199110,2cyclictest0-21swapper/012:26:060
2340199110,10cyclictest0-21swapper/008:05:260
2340599107,2cyclictest29822-21kworker/u8:108:00:013
2340599105,3cyclictest24111-21cpuspeed_turbos08:30:183
2340599102,5cyclictest0-21swapper/307:25:013
2340499107,2cyclictest0-21swapper/210:57:062
2340499105,3cyclictest30142-21grep07:25:152
2340499105,3cyclictest13551-21cron08:05:012
2340499102,6cyclictest0-21swapper/209:00:272
2340499100,6cyclictest0-21swapper/207:25:002
2340399102,6cyclictest0-21swapper/108:05:011
2340399101,3cyclictest0-21swapper/107:10:191
2340399100,8cyclictest11770-21apt-get08:00:151
2340399100,8cyclictest0-21swapper/108:45:151
2340399100,8cyclictest0-21swapper/107:30:141
2340199107,2cyclictest27178-21kworker/u8:207:50:040
2340199100,7cyclictest0-21swapper/008:01:590
2340199100,6cyclictest0-21swapper/009:08:210
234059994,3cyclictest23923-21tr07:10:153
234049996,2cyclictest0-21swapper/207:30:202
234049994,3cyclictest3521-21egrep07:40:162
234049990,8cyclictest0-21swapper/207:45:122
234019996,2cyclictest0-21swapper/008:40:220
234019996,2cyclictest0-21swapper/007:55:140
234019991,6cyclictest5129-21runrttasks07:20:160
234019991,6cyclictest0-21swapper/008:20:240
234059983,3cyclictest3505-21aten2.4-expect09:00:153
234049983,3cyclictest26496-21modprobe08:35:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional