You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 13:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Wed Nov 12, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8418211796,15sleep00-21swapper/019:05:150
9745211597,12sleep20-21swapper/219:09:162
9447211396,12sleep30-21swapper/319:05:293
9512210897,6sleep10-21swapper/119:06:201
295232980,0sleep20-21swapper/222:52:082
23312810,0sleep20-21swapper/221:18:362
282852770,1sleep1221ktimersoftd/123:24:531
52172750,0sleep00-21swapper/023:00:200
124162720,1sleep10-21swapper/123:40:261
983099330,2cyclictest0-21swapper/022:35:150
91572290,0sleep20-21swapper/220:25:242
983199221,19cyclictest0-21swapper/123:05:131
983199221,19cyclictest0-21swapper/121:25:151
983199211,18cyclictest0-21swapper/100:25:141
983199210,2cyclictest0-21swapper/121:15:131
983099211,18cyclictest0-21swapper/023:50:130
9833992017,2cyclictest0-21swapper/321:10:123
983399200,2cyclictest0-21swapper/323:50:133
983299200,18cyclictest0-21swapper/223:50:132
983199200,18cyclictest0-21swapper/123:25:141
983099200,2cyclictest0-21swapper/023:35:110
201282200,0sleep10-21swapper/123:15:241
9833991916,2cyclictest22398-21kworker/u8:100:34:023
9833991815,2cyclictest7494-21kworker/u8:222:33:043
9833991815,2cyclictest27460-21kworker/u8:122:24:053
9833991815,2cyclictest27460-21kworker/u8:122:17:043
16962180,0sleep10-21swapper/120:10:011
9833991714,2cyclictest8754-21kworker/u8:023:48:043
9833991714,2cyclictest473-21kworker/u8:000:02:063
9833991714,2cyclictest17681-21kworker/u8:021:17:043
983199171,3cyclictest0-21swapper/119:40:001
9831991713,2cyclictest0-21swapper/121:50:121
983099171,14cyclictest29579-21latency_hist22:20:000
983099171,14cyclictest0-21swapper/000:30:300
9833991613,2cyclictest7494-21kworker/u8:222:41:043
9833991613,2cyclictest28243-21kworker/u8:021:46:063
9833991613,2cyclictest24379-21kworker/u8:022:47:053
983199161,3cyclictest0-21swapper/100:10:141
9831991611,4cyclictest0-21swapper/123:45:141
983199161,13cyclictest5928-21wc23:35:011
9830991613,2cyclictest27460-21kworker/u8:122:23:040
983099161,13cyclictest0-21swapper/023:15:500
983399151,3cyclictest24484-21ssh22:14:403
9833991512,2cyclictest7494-21kworker/u8:222:35:053
9833991512,2cyclictest2940-21kworker/u8:121:53:043
9833991512,2cyclictest24379-21kworker/u8:023:05:043
983399151,12cyclictest10822-21memory19:10:183
983399150,2cyclictest0-21swapper/323:02:263
983399150,2cyclictest0-21swapper/300:25:463
983399150,13cyclictest0-21swapper/323:30:253
983399150,13cyclictest0-21swapper/322:59:083
983399150,13cyclictest0-21swapper/321:20:213
983199151,3cyclictest2437-21aten2.4_r3power22:25:131
9831991510,3cyclictest0-21swapper/123:00:181
983199150,14cyclictest0-21swapper/122:51:321
983199150,14cyclictest0-21swapper/121:42:581
983199150,13cyclictest11840-21apt-get19:15:071
983199150,13cyclictest0-21swapper/120:30:241
9830991510,3cyclictest9124-21ssh23:05:060
983399141,2cyclictest473-21iostat_ios00:35:183
9833991411,2cyclictest27460-21kworker/u8:122:27:053
9833991411,2cyclictest24876-21kworker/u8:219:37:053
9833991411,2cyclictest24379-21kworker/u8:022:52:053
9833991411,2cyclictest15949-21kworker/u8:123:56:043
9833991411,2cyclictest12337-21kworker/u8:000:18:023
9833991411,2cyclictest1131-21kworker/u8:223:42:053
9833991411,2cyclictest0-21swapper/300:24:053
983399140,2cyclictest0-21swapper/323:10:563
983399140,12cyclictest26036-21diskmemload23:15:193
983399140,12cyclictest14231-21ssh22:03:513
983299140,12cyclictest2846-21ssh22:25:152
983299140,12cyclictest0-21swapper/222:11:342
983199142,7cyclictest0-21swapper/122:20:001
983199141,3cyclictest0-21swapper/120:15:121
983199141,3cyclictest0-21swapper/120:15:111
9831991411,2cyclictest0-21swapper/100:01:311
983199140,4cyclictest0-21swapper/123:14:171
983199140,4cyclictest0-21swapper/100:05:041
983199140,3cyclictest0-21swapper/122:58:031
983199140,13cyclictest0-21swapper/121:33:111
983199140,13cyclictest0-21swapper/120:45:251
983199140,13cyclictest0-21swapper/119:20:231
983199140,12cyclictest0-21swapper/122:35:161
983199140,12cyclictest0-21swapper/122:03:351
983199140,12cyclictest0-21swapper/100:30:141
983099149,3cyclictest0-21swapper/023:22:530
983099142,7cyclictest0-21swapper/020:30:000
983099142,6cyclictest26036-21diskmemload23:15:000
983099142,6cyclictest0-21swapper/000:05:010
9830991411,2cyclictest0-21swapper/021:53:300
9830991410,2cyclictest18141-21apt-get20:50:160
9830991410,2cyclictest0-21swapper/022:30:180
983099140,2cyclictest25261-21sh22:48:040
983099140,2cyclictest0-21swapper/000:16:200
983099140,13cyclictest0-21swapper/000:20:200
983099140,12cyclictest0-21swapper/023:29:160
983099140,12cyclictest0-21swapper/021:33:490
983099140,12cyclictest0-21swapper/000:14:290
266822140,2sleep00-21swapper/023:55:210
196982140,3sleep019708-21sh22:09:470
160582140,2sleep216068-21ssh22:38:552
9833991310,2cyclictest7657-21kworker/u8:020:37:053
9833991310,2cyclictest2940-21kworker/u8:122:09:043
9833991310,2cyclictest22432-21kworker/u8:023:25:053
9833991310,2cyclictest18238-21kworker/u8:019:48:053
9833991310,2cyclictest15949-21kworker/u8:100:05:053
9833991310,2cyclictest15590-21kworker/u8:121:00:023
983399130,2cyclictest0-21swapper/321:55:063
983399130,2cyclictest0-21swapper/321:30:363
983399130,12cyclictest0-21swapper/320:05:153
983299131,11cyclictest0-21swapper/219:15:152
9832991310,2cyclictest26173-21ssh21:42:302
9832991310,2cyclictest26036-21diskmemload21:33:442
9832991310,2cyclictest0-21swapper/223:57:412
9832991310,2cyclictest0-21swapper/223:39:152
9832991310,2cyclictest0-21swapper/223:25:122
9832991310,2cyclictest0-21swapper/223:00:532
9832991310,2cyclictest0-21swapper/222:20:142
9832991310,2cyclictest0-21swapper/222:17:132
9832991310,2cyclictest0-21swapper/221:46:232
9832991310,2cyclictest0-21swapper/221:12:432
9832991310,2cyclictest0-21swapper/220:40:222
9832991310,2cyclictest0-21swapper/200:29:302
9832991310,2cyclictest0-21swapper/200:00:042
983299130,2cyclictest0-21swapper/222:08:042
983299130,2cyclictest0-21swapper/200:19:432
983299130,12cyclictest0-21swapper/223:10:352
983299130,12cyclictest0-21swapper/222:47:142
983299130,12cyclictest0-21swapper/220:20:252
983299130,11cyclictest0-21swapper/223:44:022
983299130,11cyclictest0-21swapper/223:20:192
983299130,11cyclictest0-21swapper/222:40:542
983199139,3cyclictest0-21swapper/122:30:191
983199138,3cyclictest0-21swapper/119:10:161
983199134,7cyclictest5911-21gdm-simple-gree20:03:591
983199132,6cyclictest0-21swapper/122:45:011
9831991310,2cyclictest0-21swapper/119:50:121
9831991310,2cyclictest0-21swapper/119:40:061
9831991310,2cyclictest0-21swapper/100:16:421
983199130,2cyclictest0-21swapper/122:20:031
983199130,12cyclictest0-21swapper/123:57:021
983199130,12cyclictest0-21swapper/123:37:441
983199130,12cyclictest0-21swapper/122:10:191
983199130,12cyclictest0-21swapper/121:45:051
983199130,12cyclictest0-21swapper/121:22:591
983199130,12cyclictest0-21swapper/121:11:311
983199130,12cyclictest0-21swapper/100:35:161
983199130,11cyclictest26036-21diskmemload22:07:541
983199130,11cyclictest26036-21diskmemload21:56:281
983199130,11cyclictest0-21swapper/122:45:271
983199130,11cyclictest0-21swapper/119:30:121
983099139,2cyclictest17052-21sh22:40:050
983099139,2cyclictest0-21swapper/021:45:000
983099138,3cyclictest4427-21sh22:26:130
983099138,3cyclictest0-21swapper/020:40:220
9830991310,2cyclictest0-21swapper/021:38:050
9830991310,2cyclictest0-21swapper/000:01:320
983099130,2cyclictest0-21swapper/023:46:040
983099130,2cyclictest0-21swapper/022:12:490
983099130,2cyclictest0-21swapper/019:25:110
983099130,2cyclictest0-21swapper/000:35:420
983099130,1cyclictest0-21swapper/019:20:200
983099130,12cyclictest0-21swapper/021:43:400
983099130,12cyclictest0-21swapper/021:26:130
983099130,11cyclictest8060-21apt-get20:25:120
983099130,11cyclictest26036-21diskmemload21:10:240
9662130,2sleep00-21swapper/021:16:320
305372130,2sleep20-21swapper/200:32:592
28282130,2sleep00-21swapper/023:30:430
983399129,2cyclictest6291-21kworker/u8:200:11:043
983399129,2cyclictest28341-21kworker/u8:220:27:043
983399129,2cyclictest24876-21kworker/u8:219:16:053
983399129,2cyclictest23840-21kworker/u8:221:06:053
983399129,2cyclictest14116-21kworker/u8:119:28:053
983399129,2cyclictest0-21swapper/323:20:553
983399129,2cyclictest0-21swapper/321:25:363
983399129,2cyclictest0-21swapper/320:30:213
983399120,2cyclictest18801-21ssh21:35:173
983399120,2cyclictest0-21swapper/320:40:203
983399120,2cyclictest0-21swapper/320:00:283
983399120,11cyclictest9028-21ssh23:37:283
983399120,11cyclictest0-21swapper/321:40:463
983399120,11cyclictest0-21swapper/319:20:183
983299129,2cyclictest1834-21apt-get20:10:132
983299129,2cyclictest0-21swapper/223:30:132
983299129,2cyclictest0-21swapper/223:05:122
983299129,2cyclictest0-21swapper/221:35:252
983299129,2cyclictest0-21swapper/200:35:202
983299129,2cyclictest0-21swapper/200:11:512
983299129,2cyclictest0-21swapper/200:05:042
983299122,7cyclictest0-21swapper/220:47:022
983299120,2cyclictest7989-21ssh21:56:452
983299120,2cyclictest0-21swapper/222:30:142
983299120,2cyclictest0-21swapper/219:20:242
983299120,2cyclictest0-21swapper/200:21:132
983299120,11cyclictest18419-21ssh23:46:552
983299120,11cyclictest0-21swapper/222:56:512
983299120,11cyclictest0-21swapper/222:00:532
983299120,11cyclictest0-21swapper/221:50:232
983299120,11cyclictest0-21swapper/221:25:122
983299120,11cyclictest0-21swapper/221:20:432
983299120,11cyclictest0-21swapper/220:35:292
983299120,11cyclictest0-21swapper/220:15:242
983299120,11cyclictest0-21swapper/220:15:232
983299120,11cyclictest0-21swapper/219:57:412
983299120,11cyclictest0-21swapper/219:50:132
983299120,11cyclictest0-21swapper/219:45:232
983199127,3cyclictest28240-21munin-node1
983199127,3cyclictest0-21swapper/121:35:161
983199121,3cyclictest0-21swapper/119:45:281
983199120,2cyclictest0-21swapper/121:00:191
983199120,11cyclictest0-21swapper/123:50:171
983199120,11cyclictest0-21swapper/120:55:191
983199120,11cyclictest0-21swapper/120:35:291
983199120,10cyclictest19177-21grep00:20:281
983199120,10cyclictest0-21swapper/121:05:221
983099129,2cyclictest5375-21kworker/u8:121:21:040
983099129,2cyclictest11671-21kworker/u8:019:15:050
983099129,2cyclictest0-21swapper/022:57:080
983099129,2cyclictest0-21swapper/000:25:000
983099122,7cyclictest0-21swapper/019:13:590
983099121,6cyclictest0-21swapper/020:00:010
983099120,7cyclictest0-21swapper/021:00:060
983099120,2cyclictest0-21swapper/022:00:160
983099120,2cyclictest0-21swapper/021:55:530
983099120,2cyclictest0-21swapper/021:05:040
983099120,11cyclictest0-21swapper/023:40:370
983099120,11cyclictest0-21swapper/022:50:380
983099120,11cyclictest0-21swapper/020:55:110
983099120,11cyclictest0-21swapper/020:05:250
983399118,2cyclictest5423-21kworker/u8:120:20:053
983399118,2cyclictest28341-21kworker/u8:219:56:043
983399118,2cyclictest18238-21kworker/u8:020:16:053
983399118,2cyclictest18238-21kworker/u8:020:16:043
983399118,2cyclictest18238-21kworker/u8:019:50:053
983399118,2cyclictest15590-21kworker/u8:120:53:043
983399118,2cyclictest12220-21kworker/u8:220:46:023
983399118,2cyclictest0-21swapper/320:10:123
983299118,2cyclictest0-21swapper/220:30:232
983299112,6cyclictest0-21swapper/221:08:582
983299110,10cyclictest0-21swapper/223:15:122
983199118,2cyclictest0-21swapper/120:40:221
983199117,2cyclictest0-21swapper/120:25:191
983199111,3cyclictest0-21swapper/120:20:151
983199110,9cyclictest0-21swapper/120:10:121
983199110,3cyclictest0-21swapper/120:50:221
983099118,2cyclictest0-21swapper/020:35:210
983099112,7cyclictest0-21swapper/020:15:590
983099112,7cyclictest0-21swapper/020:15:590
983099110,1cyclictest0-21swapper/020:10:180
983099110,10cyclictest0-21swapper/020:45:140
983399107,2cyclictest0-21swapper/320:55:183
983399105,3cyclictest18620-21fschecks_time19:30:143
983399102,5cyclictest0-21swapper/319:41:013
983299107,2cyclictest0-21swapper/221:00:152
983299106,2cyclictest0-21swapper/220:55:142
983299102,6cyclictest0-21swapper/219:40:252
983299101,6cyclictest0-21swapper/219:30:232
983299101,5cyclictest0-21swapper/219:10:132
983199107,2cyclictest0-21swapper/119:25:121
983099105,3cyclictest22749-21fschecks_time19:40:160
983099105,3cyclictest20233-21aten2.4_r3power19:35:170
983099104,4cyclictest32056-21cron20:05:000
983099100,2cyclictest0-21swapper/019:30:510
98329996,2cyclictest0-21swapper/220:05:162
98329996,2cyclictest0-21swapper/220:00:012
98329996,2cyclictest0-21swapper/219:25:242
98309996,2cyclictest0-21swapper/019:50:140
98309991,6cyclictest0-21swapper/020:20:260
98309991,5cyclictest0-21swapper/019:45:140
98329983,3cyclictest21059-21open_files19:35:222
98329981,4cyclictest0-21swapper/220:50:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional