You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 12:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Fri Nov 28, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20221211697,14sleep30-21swapper/319:09:073
202202115104,6sleep20-21swapper/219:09:062
20132210695,6sleep10-21swapper/119:07:561
3243721050,1sleep332436-21ssh23:57:283
19968210078,17sleep00-21swapper/019:05:510
113232740,0sleep00-21swapper/023:00:170
30742690,1sleep3381ktimersoftd/321:05:203
120042690,0sleep20-21swapper/223:00:272
310842660,4sleep32032299cyclictest22:12:263
132402580,0sleep20-21swapper/223:36:392
2032199211,18cyclictest0-21swapper/222:09:042
2032199210,4cyclictest0-21swapper/223:22:042
2032099210,16cyclictest0-21swapper/122:49:041
2032199202,2cyclictest0-21swapper/222:27:042
218782190,1sleep3381ktimersoftd/320:30:273
2032199192,2cyclictest0-21swapper/223:49:042
20321991915,3cyclictest2218-21kworker/u8:220:11:042
20321991915,2cyclictest23805-21kworker/u8:222:51:042
2032199190,2cyclictest0-21swapper/221:28:012
2031999190,18cyclictest0-21swapper/022:29:040
20321991815,2cyclictest24295-21kworker/u8:019:39:042
20321991815,2cyclictest12042-21kworker/u8:119:27:032
20321991714,2cyclictest24517-21kworker/u8:120:55:042
20321991714,2cyclictest18826-21kworker/u8:022:00:042
20321991714,2cyclictest15182-21kworker/u8:223:11:042
2032099170,15cyclictest0-21swapper/123:26:121
20321991613,2cyclictest32160-21kworker/u8:119:41:042
20321991613,2cyclictest24517-21kworker/u8:121:06:042
20321991613,2cyclictest24295-21kworker/u8:020:00:042
20321991613,2cyclictest24295-21kworker/u8:019:48:042
20321991613,2cyclictest24273-21kworker/u8:122:43:032
2032199160,2cyclictest0-21swapper/222:21:042
2032199160,14cyclictest0-21swapper/221:56:042
2032099162,2cyclictest0-21swapper/121:25:011
20320991613,2cyclictest20865-21kworker/u8:100:31:041
2032099161,13cyclictest15273-21apt-get23:05:111
2032099161,13cyclictest10458-21df22:25:141
2032099161,13cyclictest0-21swapper/123:10:131
2032099161,13cyclictest0-21swapper/122:33:141
2032099160,2cyclictest0-21swapper/100:22:011
20319991613,2cyclictest32573-21kworker/u8:023:22:040
2031999160,3cyclictest16237-21sendmail20:20:000
16842161,3sleep00-21swapper/000:34:300
20322991512,2cyclictest20865-21kworker/u8:100:22:013
20322991512,2cyclictest0-21swapper/322:04:073
2032299150,13cyclictest2479-21aten2.4_r3power00:35:133
2032299150,13cyclictest0-21swapper/323:44:573
2032299150,13cyclictest0-21swapper/322:50:173
2032299150,13cyclictest0-21swapper/321:14:073
20321991512,2cyclictest8475-21kworker/u8:120:05:042
20321991512,2cyclictest31894-21kworker/u8:000:06:032
20321991512,2cyclictest28633-21kworker/u8:219:31:042
20321991512,2cyclictest27693-21kworker/u8:121:54:042
20321991512,2cyclictest24517-21kworker/u8:121:20:042
20321991512,2cyclictest24295-21kworker/u8:019:20:012
20321991512,2cyclictest23791-21kworker/u8:123:50:042
20321991512,2cyclictest2218-21kworker/u8:220:47:042
20321991512,2cyclictest20602-21kworker/u8:219:19:042
20321991512,2cyclictest14597-21kworker/u8:020:37:042
2032099159,4cyclictest21-21rcuc/121:48:041
20320991512,2cyclictest11436-21kworker/u8:200:15:041
2032099150,2cyclictest0-21swapper/123:43:051
2032099150,2cyclictest0-21swapper/121:40:291
2032099150,2cyclictest0-21swapper/121:35:201
2032099150,2cyclictest0-21swapper/121:24:301
20319991512,2cyclictest23805-21kworker/u8:222:09:040
20319991510,3cyclictest26370-21ntp_states00:25:210
84882140,3sleep10-21swapper/120:00:131
74692140,3sleep07477-21ssh22:21:140
20322991411,2cyclictest21775-21ssh23:45:273
20322991411,2cyclictest0-21swapper/322:18:273
20322991411,2cyclictest0-21swapper/321:19:473
20322991411,2cyclictest0-21swapper/320:15:173
20322991411,2cyclictest0-21swapper/300:15:073
2032299140,2cyclictest0-21swapper/322:37:073
2032299140,2cyclictest0-21swapper/321:55:173
2032299140,13cyclictest0-21swapper/323:25:273
2032299140,13cyclictest0-21swapper/322:46:573
2032299140,13cyclictest0-21swapper/322:41:473
2032299140,12cyclictest3983-21diskmemload21:25:203
2032299140,12cyclictest3983-21diskmemload00:29:273
2032299140,12cyclictest10016-21ssh22:59:403
2032299140,12cyclictest0-21swapper/323:50:583
2032299140,12cyclictest0-21swapper/323:35:123
2032299140,12cyclictest0-21swapper/323:30:193
2032299140,12cyclictest0-21swapper/321:30:253
2032299140,12cyclictest0-21swapper/320:10:153
2032299140,12cyclictest0-21swapper/300:32:393
20321991411,2cyclictest6752-21kworker/u8:221:31:042
20321991411,2cyclictest4727-21kworker/u8:023:32:042
20321991411,2cyclictest23805-21kworker/u8:222:31:042
20321991411,2cyclictest23791-21kworker/u8:100:04:012
20321991411,2cyclictest2218-21kworker/u8:220:25:042
20321991411,2cyclictest14597-21kworker/u8:020:19:042
20321991411,2cyclictest0-21swapper/223:05:272
20321991410,3cyclictest5007-21munin-run22:20:002
20321991410,3cyclictest11740-21df_abs00:10:142
20321991410,2cyclictest0-21swapper/200:30:062
2032199140,2cyclictest0-21swapper/222:35:042
2032199140,2cyclictest0-21swapper/222:13:042
2032199140,2cyclictest0-21swapper/221:44:052
2032199140,13cyclictest0-21swapper/219:55:182
2032199140,12cyclictest0-21swapper/223:42:042
2032099148,5cyclictest3983-21diskmemload00:36:161
2032099148,4cyclictest0-21swapper/121:31:431
20320991411,2cyclictest0-21swapper/123:45:291
20320991411,2cyclictest0-21swapper/123:02:291
20320991411,2cyclictest0-21swapper/122:56:191
20320991411,2cyclictest0-21swapper/122:19:011
20320991411,2cyclictest0-21swapper/121:19:201
20320991411,2cyclictest0-21swapper/100:01:121
2032099140,2cyclictest0-21swapper/122:52:591
2032099140,2cyclictest0-21swapper/121:58:501
2032099140,2cyclictest0-21swapper/121:51:201
2032099140,13cyclictest3983-21diskmemload23:33:291
2032099140,13cyclictest0-21swapper/123:37:191
2032099140,13cyclictest0-21swapper/123:22:291
2032099140,13cyclictest0-21swapper/122:13:191
2032099140,13cyclictest0-21swapper/119:50:201
2032099140,13cyclictest0-21swapper/100:10:291
2032099140,13cyclictest0-21swapper/100:09:591
2032099140,12cyclictest3983-21diskmemload23:15:191
2032099140,12cyclictest3983-21diskmemload21:10:231
2032099140,12cyclictest0-21swapper/122:20:341
2032099140,12cyclictest0-21swapper/120:35:141
2031999141,2cyclictest19492-21ssh00:18:310
20319991411,2cyclictest6752-21kworker/u8:221:28:010
20319991411,2cyclictest23791-21kworker/u8:123:49:040
2031999140,12cyclictest710-21latency_hist22:50:000
2031999140,12cyclictest32194-21ssh23:57:020
2031999140,12cyclictest0-21swapper/022:55:210
2031999140,12cyclictest0-21swapper/021:50:150
2031999140,12cyclictest0-21swapper/020:35:000
25262130,2sleep20-21swapper/223:25:212
20322991310,2cyclictest5869-21kworker/u8:123:00:043
20322991310,2cyclictest21713-21ssh23:10:493
20322991310,2cyclictest0-21swapper/323:20:193
20322991310,2cyclictest0-21swapper/323:05:183
20322991310,2cyclictest0-21swapper/321:50:063
20322991310,2cyclictest0-21swapper/321:45:593
20322991310,2cyclictest0-21swapper/321:44:573
20322991310,2cyclictest0-21swapper/321:21:353
20322991310,2cyclictest0-21swapper/319:55:153
20322991310,2cyclictest0-21swapper/300:11:473
2032299130,2cyclictest0-21swapper/322:05:573
2032299130,2cyclictest0-21swapper/320:35:133
2032299130,2cyclictest0-21swapper/300:06:423
2032299130,12cyclictest7831-21ssh22:21:493
2032299130,12cyclictest0-21swapper/323:19:333
2032299130,12cyclictest0-21swapper/322:32:593
2032299130,12cyclictest0-21swapper/321:38:593
2032299130,12cyclictest0-21swapper/320:40:253
2032299130,11cyclictest25903-21unixbench_multi19:20:283
2032299130,11cyclictest0-21swapper/300:00:113
20321991310,2cyclictest32619-21kworker/u8:221:04:042
20321991310,2cyclictest2255-21sh00:35:042
2032199130,2cyclictest0-21swapper/221:35:172
2032199130,2cyclictest0-21swapper/221:19:042
2032199130,2cyclictest0-21swapper/200:26:552
2032199130,12cyclictest0-21swapper/221:11:182
2032199130,12cyclictest0-21swapper/200:20:172
2032199130,11cyclictest0-21swapper/222:55:132
2032099131,11cyclictest0-21swapper/122:35:491
2032099131,11cyclictest0-21swapper/119:20:231
20320991310,2cyclictest0-21swapper/122:01:171
20320991310,2cyclictest0-21swapper/120:50:161
2032099130,2cyclictest0-21swapper/123:55:431
2032099130,2cyclictest0-21swapper/119:35:261
2032099130,12cyclictest3983-21diskmemload23:52:001
2032099130,12cyclictest3983-21diskmemload22:05:571
2032099130,12cyclictest0-21swapper/122:40:261
2032099130,12cyclictest0-21swapper/120:10:121
2032099130,12cyclictest0-21swapper/119:30:111
2032099130,12cyclictest0-21swapper/100:25:511
2031999131,2cyclictest5129-21runrttasks23:35:510
20319991310,2cyclictest0-21swapper/020:55:130
20319991310,2cyclictest0-21swapper/000:36:470
2031999130,6cyclictest0-21swapper/023:10:120
259502120,2sleep10-21swapper/120:40:271
2032299129,2cyclictest0-21swapper/322:25:203
2032299129,2cyclictest0-21swapper/319:30:123
2032299120,11cyclictest24517-21kworker/u8:121:00:043
2032299120,11cyclictest0-21swapper/319:40:263
2032199129,2cyclictest24295-21kworker/u8:019:53:042
2032199120,11cyclictest0-21swapper/222:45:152
2032199120,11cyclictest0-21swapper/220:40:192
2032199120,11cyclictest0-21swapper/220:30:172
2032199120,10cyclictest0-21swapper/223:18:042
2032099128,2cyclictest0-21swapper/119:40:141
2032099122,9cyclictest24177-21kworker/1:019:55:131
2032099121,9cyclictest0-21swapper/119:45:221
2032099121,3cyclictest3991-21latency_hist21:10:001
2032099121,3cyclictest30346-21apt-get20:55:061
2032099121,3cyclictest24349-21munin-run19:20:001
2032099120,2cyclictest0-21swapper/120:05:151
2032099120,11cyclictest0-21swapper/120:20:191
2031999129,2cyclictest3983-21diskmemload23:41:230
2031999129,2cyclictest10009-21ssh00:08:200
2031999129,2cyclictest0-21swapper/022:50:330
2031999129,2cyclictest0-21swapper/022:36:110
2031999129,2cyclictest0-21swapper/022:31:220
2031999129,2cyclictest0-21swapper/022:15:200
2031999129,2cyclictest0-21swapper/021:41:000
2031999129,2cyclictest0-21swapper/021:37:220
2031999129,2cyclictest0-21swapper/021:20:160
2031999129,2cyclictest0-21swapper/021:16:320
2031999129,2cyclictest0-21swapper/020:45:000
2031999120,2cyclictest0-21swapper/023:25:410
2031999120,2cyclictest0-21swapper/021:32:530
2031999120,2cyclictest0-21swapper/021:10:220
2031999120,2cyclictest0-21swapper/000:12:510
2031999120,11cyclictest0-21swapper/023:51:500
2031999120,11cyclictest0-21swapper/023:18:520
2031999120,11cyclictest0-21swapper/022:41:520
2031999120,11cyclictest0-21swapper/022:02:120
2031999120,11cyclictest0-21swapper/021:55:120
2031999120,11cyclictest0-21swapper/000:22:520
2031999120,11cyclictest0-21swapper/000:00:410
161752121,7sleep216403-21sed00:15:162
2032299118,2cyclictest0-21swapper/319:45:123
2032199118,2cyclictest3983-21diskmemload23:59:062
2032199118,2cyclictest0-21swapper/221:45:252
2032199118,2cyclictest0-21swapper/220:20:152
2032199112,7cyclictest0-21swapper/220:50:022
2032199110,2cyclictest0-21swapper/219:11:042
2032099117,2cyclictest0-21swapper/119:25:121
2032099111,9cyclictest32629-21munin-node1
2032099110,3cyclictest0-21swapper/120:45:151
2032099110,10cyclictest0-21swapper/120:25:221
2031999118,2cyclictest3983-21diskmemload23:30:190
2031999118,2cyclictest0-21swapper/019:25:160
2031999110,2cyclictest0-21swapper/022:11:320
161922110,1sleep00-21swapper/023:05:190
2032299107,2cyclictest0-21swapper/320:45:273
2032299107,2cyclictest0-21swapper/320:25:143
2032299107,2cyclictest0-21swapper/319:10:163
2032299106,2cyclictest10782-21df_abs20:05:153
2032299105,3cyclictest24127-21sshd19:17:233
2032099107,2cyclictest12424-21kworker/u8:019:10:041
2032099105,3cyclictest14516-21aten2.4_r3power20:15:131
2031999105,3cyclictest24311-21sendmail20:40:000
2031999100,9cyclictest32619-21kworker/u8:221:01:040
2031999100,2cyclictest0-21swapper/021:45:440
852290,1sleep00-21swapper/019:40:160
203229996,2cyclictest16813-21df_abs20:20:153
203229996,2cyclictest0-21swapper/320:55:153
203229996,2cyclictest0-21swapper/320:02:423
203229996,2cyclictest0-21swapper/319:25:133
203229990,7cyclictest24295-21kworker/u8:019:35:043
203209991,6cyclictest0-21swapper/120:30:151
203199996,2cyclictest27833-21cat20:45:270
203199996,2cyclictest0-21swapper/019:30:210
203199996,2cyclictest0-21swapper/019:21:330
203199994,3cyclictest5129-21runrttasks19:51:120
203199990,2cyclictest0-21swapper/020:26:310
3840280,1sleep00-21swapper/019:48:140
203229983,3cyclictest4347-21cut19:50:153
203229980,7cyclictest0-21swapper/320:50:293
203199985,2cyclictest22547-21apt-get19:15:120
203199985,2cyclictest16623-21aten2.4_r3power20:20:140
203199985,2cyclictest13828-21unixbench_singl20:10:260
203199984,3cyclictest21585-21expr19:10:240
203199984,2cyclictest4324-21nscd19:35:160
203199983,3cyclictest8661-21tr20:00:120
203199983,3cyclictest6375-21seq19:55:100
203199981,2cyclictest3075-21cat21:05:190
203199980,6cyclictest0-21swapper/020:50:260
203199974,2cyclictest10483-21telnet20:05:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional