You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 03:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Tue Dec 02, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8198211697,14sleep20-21swapper/219:09:002
81512116105,6sleep00-21swapper/019:08:230
6889211199,6sleep10-21swapper/119:05:141
8154211094,11sleep30-21swapper/319:08:253
94792880,1sleep00-21swapper/000:20:130
123542880,0sleep10-21swapper/100:23:181
86662810,0sleep10-21swapper/123:10:151
165542800,1sleep30-21swapper/322:10:133
325522780,2sleep3381ktimersoftd/300:10:183
299212710,1sleep00-21swapper/021:15:140
69812650,0sleep20-21swapper/222:00:102
168722230,0sleep10-21swapper/121:35:201
831099221,2cyclictest0-21swapper/322:55:123
831099221,19cyclictest0-21swapper/323:20:113
831099212,7cyclictest0-21swapper/323:30:583
159062210,0sleep10-21swapper/100:26:271
830799201,2cyclictest13182-21ssh23:50:110
8309991916,2cyclictest721-21kworker/u8:222:31:042
8309991916,2cyclictest2904-21kworker/u8:100:18:052
830899190,2cyclictest0-21swapper/122:15:121
830799190,18cyclictest0-21swapper/000:04:020
831099180,2cyclictest0-21swapper/322:15:123
8309991815,2cyclictest24319-21kworker/u8:223:30:052
8309991815,2cyclictest18416-21kworker/u8:122:20:012
8308991814,2cyclictest0-21swapper/122:20:131
830899180,3cyclictest0-21swapper/100:14:591
29902180,1sleep20-21swapper/200:13:422
243192180,1sleep124320-21sh21:44:091
8310991713,3cyclictest31564-21kworker/3:123:50:113
8309991714,2cyclictest5984-21kworker/u8:023:44:042
8309991714,2cyclictest5984-21kworker/u8:023:44:042
8309991714,2cyclictest16743-21kworker/u8:223:58:042
8309991713,2cyclictest0-21swapper/220:00:152
830999170,15cyclictest0-21swapper/222:49:042
8308991712,3cyclictest31080-21turbostat23:39:551
8308991712,3cyclictest0-21swapper/100:37:541
8309991613,2cyclictest26315-21kworker/u8:121:51:042
8309991613,2cyclictest26281-21kworker/u8:223:09:042
8309991613,2cyclictest24238-21kworker/u8:023:01:052
830999160,2cyclictest0-21swapper/223:18:012
831099152,6cyclictest0-21swapper/323:18:013
8310991512,2cyclictest0-21swapper/323:42:553
8310991512,2cyclictest0-21swapper/323:42:553
8310991512,2cyclictest0-21swapper/323:35:463
8309991512,2cyclictest26315-21kworker/u8:121:56:042
8309991512,2cyclictest18416-21kworker/u8:122:12:052
830999151,12cyclictest0-21swapper/221:45:012
8309991510,3cyclictest22741-21ssh21:41:332
8309991510,3cyclictest0-21swapper/221:30:042
830999150,14cyclictest0-21swapper/221:28:082
830999150,13cyclictest0-21swapper/221:20:082
830999150,13cyclictest0-21swapper/221:20:082
830899150,13cyclictest2025-21ssh22:29:261
830899150,13cyclictest0-21swapper/123:50:031
8307991512,2cyclictest0-21swapper/022:21:480
830799151,12cyclictest31176-21munin-node0
830799150,14cyclictest0-21swapper/023:10:010
830799150,14cyclictest0-21swapper/019:10:050
830799150,13cyclictest24418-21diskmemload22:30:180
8310991411,2cyclictest0-21swapper/300:20:463
8310991411,2cyclictest0-21swapper/300:17:253
831099140,13cyclictest0-21swapper/300:07:453
831099140,12cyclictest0-21swapper/300:25:463
830999149,3cyclictest31256-21ssh21:16:002
8309991411,2cyclictest26281-21kworker/u8:222:58:012
8309991411,2cyclictest24319-21kworker/u8:223:36:012
8309991411,2cyclictest15694-21kworker/u8:300:37:052
8309991411,2cyclictest0-21swapper/222:18:582
8309991411,2cyclictest0-21swapper/221:36:092
8309991411,2cyclictest0-21swapper/200:23:582
8309991411,2cyclictest0-21swapper/200:08:572
830999140,13cyclictest0-21swapper/222:50:582
830999140,12cyclictest19561-21proc_pri19:35:192
830999140,12cyclictest0-21swapper/223:11:042
830999140,12cyclictest0-21swapper/222:08:042
830899141,3cyclictest24418-21diskmemload00:15:121
830899141,2cyclictest16575-21apt-get20:50:141
8308991411,2cyclictest24418-21diskmemload21:56:531
8308991411,2cyclictest24418-21diskmemload00:01:431
8308991411,2cyclictest0-21swapper/123:30:231
8308991411,2cyclictest0-21swapper/122:04:341
8308991411,2cyclictest0-21swapper/121:29:541
830899140,13cyclictest0-21swapper/122:11:231
830899140,12cyclictest24418-21diskmemload22:31:161
830899140,12cyclictest24418-21diskmemload22:05:151
830899140,12cyclictest0-21swapper/121:45:051
8307991411,2cyclictest24418-21diskmemload21:51:190
8307991411,2cyclictest17691-21smartctl20:50:250
8307991411,2cyclictest0-21swapper/000:29:190
8307991410,2cyclictest0-21swapper/023:45:160
830799140,2cyclictest5046-21missed_timers00:15:180
830799140,2cyclictest4029-21ssh21:55:580
830799140,2cyclictest24418-21diskmemload22:18:590
830799140,2cyclictest0-21swapper/021:46:080
830799140,2cyclictest0-21swapper/021:35:480
830799140,2cyclictest0-21swapper/020:40:190
830799140,2cyclictest0-21swapper/019:45:180
830799140,13cyclictest0-21swapper/021:30:380
830799140,13cyclictest0-21swapper/000:31:380
830799140,12cyclictest27460-21memory19:55:180
830799140,12cyclictest19494-21sh23:21:210
830799140,12cyclictest0-21swapper/022:55:380
830799140,12cyclictest0-21swapper/022:14:350
830799140,12cyclictest0-21swapper/022:04:300
830799140,12cyclictest0-21swapper/021:10:120
193592140,2sleep10-21swapper/122:46:511
831099132,6cyclictest0-21swapper/320:35:003
831099132,6cyclictest0-21swapper/320:30:013
831099132,6cyclictest0-21swapper/320:15:003
831099132,6cyclictest0-21swapper/319:45:003
831099131,7cyclictest22842-21copy00:35:003
831099131,7cyclictest0-21swapper/319:50:003
8310991310,2cyclictest8306-21cyclictest21:15:143
831099130,2cyclictest0-21swapper/300:36:063
831099130,10cyclictest0-21swapper/321:37:553
8309991310,2cyclictest6597-21kworker/u8:220:27:052
8309991310,2cyclictest6091-21kworker/u8:019:26:052
8309991310,2cyclictest24319-21kworker/u8:223:27:052
8309991310,2cyclictest24298-21kworker/u8:020:09:042
8309991310,2cyclictest18416-21kworker/u8:122:26:052
8309991310,2cyclictest16641-21kworker/u8:220:54:022
8309991310,2cyclictest10029-21kworker/u8:119:47:042
8309991310,2cyclictest0-21swapper/223:20:172
8309991310,2cyclictest0-21swapper/220:30:282
830999130,2cyclictest0-21swapper/222:40:122
830999130,2cyclictest0-21swapper/220:45:282
830999130,1cyclictest0-21swapper/222:36:392
830999130,11cyclictest0-21swapper/219:14:592
830999130,11cyclictest0-21swapper/200:32:392
830899132,9cyclictest21-21rcuc/120:25:151
8308991310,2cyclictest0-21swapper/123:48:151
8308991310,2cyclictest0-21swapper/123:23:161
8308991310,2cyclictest0-21swapper/123:01:231
8308991310,2cyclictest0-21swapper/122:55:241
8308991310,2cyclictest0-21swapper/122:50:261
830899130,2cyclictest24418-21diskmemload22:42:541
830899130,2cyclictest0-21swapper/123:25:341
830899130,2cyclictest0-21swapper/123:16:441
830899130,2cyclictest0-21swapper/121:34:351
830899130,12cyclictest0-21swapper/123:41:231
830899130,12cyclictest0-21swapper/123:41:231
830899130,12cyclictest0-21swapper/123:05:141
830899130,12cyclictest0-21swapper/122:39:131
830899130,12cyclictest0-21swapper/121:50:551
830899130,12cyclictest0-21swapper/121:16:141
830899130,12cyclictest0-21swapper/100:30:031
830899130,11cyclictest18404-21turbostat19:34:591
830799139,2cyclictest0-21swapper/020:40:000
8307991310,2cyclictest0-21swapper/023:43:520
8307991310,2cyclictest0-21swapper/023:43:510
8307991310,2cyclictest0-21swapper/023:26:010
8307991310,2cyclictest0-21swapper/023:15:170
8307991310,2cyclictest0-21swapper/023:05:510
8307991310,2cyclictest0-21swapper/022:53:510
8307991310,2cyclictest0-21swapper/022:48:080
8307991310,2cyclictest0-21swapper/022:41:410
8307991310,2cyclictest0-21swapper/021:25:370
8307991310,2cyclictest0-21swapper/021:20:090
8307991310,2cyclictest0-21swapper/021:20:090
8307991310,2cyclictest0-21swapper/020:05:180
8307991310,2cyclictest0-21swapper/019:15:130
830799130,2cyclictest0-21swapper/023:00:480
830799130,12cyclictest23775-21ssh00:35:170
830799130,12cyclictest0-21swapper/023:56:080
830799130,12cyclictest0-21swapper/023:35:410
830799130,12cyclictest0-21swapper/023:34:010
830799130,12cyclictest0-21swapper/022:05:160
830799130,12cyclictest0-21swapper/021:40:580
153422130,2sleep20-21swapper/200:25:332
106662130,2sleep024418-21diskmemload22:37:560
831099129,2cyclictest0-21swapper/300:04:453
831099122,7cyclictest0-21swapper/321:08:583
831099122,7cyclictest0-21swapper/320:59:583
831099121,9cyclictest0-21swapper/321:05:013
831099121,3cyclictest0-21swapper/323:25:023
830999129,2cyclictest6091-21kworker/u8:019:18:012
830999129,2cyclictest28785-21kworker/u8:120:14:042
830999129,2cyclictest22247-21kworker/u8:021:10:042
830999129,2cyclictest10029-21kworker/u8:119:21:042
830999129,2cyclictest0-21swapper/223:45:032
830999120,11cyclictest0-21swapper/223:50:262
830999120,11cyclictest0-21swapper/220:35:232
830999120,11cyclictest0-21swapper/219:55:202
830899129,2cyclictest0-21swapper/123:55:431
830899129,2cyclictest0-21swapper/121:20:541
830899129,2cyclictest0-21swapper/121:20:541
830899129,2cyclictest0-21swapper/119:20:131
830899124,5cyclictest332250irq/24-eth120:40:001
830899122,5cyclictest0-21swapper/119:20:011
830899121,9cyclictest8289-21apt-get19:10:061
830899121,6cyclictest0-21swapper/120:05:001
830899120,2cyclictest24418-21diskmemload21:10:071
830899120,2cyclictest0-21swapper/100:08:541
830899120,11cyclictest0-21swapper/119:35:251
830899120,11cyclictest0-21swapper/119:25:181
830899120,10cyclictest0-21swapper/120:40:221
830799129,2cyclictest0-21swapper/019:35:250
830799129,2cyclictest0-21swapper/019:20:170
830799129,2cyclictest0-21swapper/000:05:040
830799120,11cyclictest0-21swapper/021:05:130
830799120,11cyclictest0-21swapper/020:45:250
830799120,11cyclictest0-21swapper/019:40:160
830799120,11cyclictest0-21swapper/000:10:390
299982120,2sleep30-21swapper/322:24:483
241852120,2sleep20-21swapper/200:00:352
141132120,2sleep30-21swapper/322:06:523
831099118,2cyclictest0-21swapper/319:10:123
831099112,7cyclictest0-21swapper/320:15:583
831099111,7cyclictest10085-21ssh22:36:593
831099111,6cyclictest16360-21munin-run20:50:003
831099111,6cyclictest0-21swapper/321:15:003
831099110,6cyclictest0-21swapper/319:55:003
831099110,10cyclictest0-21swapper/321:23:363
831099110,10cyclictest0-21swapper/321:23:353
830999118,2cyclictest24298-21kworker/u8:020:42:042
830999118,2cyclictest24298-21kworker/u8:020:21:042
830999118,2cyclictest16641-21kworker/u8:220:55:042
830999112,7cyclictest0-21swapper/219:40:582
830999110,10cyclictest0-21swapper/219:50:182
830899118,2cyclictest0-21swapper/119:45:231
830899111,8cyclictest327-21head20:10:131
830899111,5cyclictest0-21swapper/121:00:001
830899111,5cyclictest0-21swapper/119:45:011
830899110,9cyclictest14563-21munin-node1
830899110,9cyclictest0-21swapper/120:15:181
830899110,10cyclictest0-21swapper/119:55:221
830799118,2cyclictest0-21swapper/020:15:170
830799118,2cyclictest0-21swapper/019:30:270
830799110,1cyclictest0-21swapper/020:00:170
830799110,1cyclictest0-21swapper/019:25:190
830799110,10cyclictest0-21swapper/019:54:590
199502110,2sleep30-21swapper/322:47:493
146762110,2sleep314686-21ssh21:33:413
831099107,2cyclictest0-21swapper/322:53:433
831099107,2cyclictest0-21swapper/321:55:093
831099107,2cyclictest0-21swapper/321:27:113
831099104,4cyclictest4324-21nscd23:45:233
831099104,4cyclictest11415-21tr19:15:173
831099103,6cyclictest0-21swapper/323:10:243
831099102,7cyclictest0-21swapper/319:58:583
831099101,7cyclictest20420-21munin-run19:40:003
831099101,3cyclictest69822sh22:00:103
831099101,3cyclictest24225-21ssh21:43:593
830999107,2cyclictest0-21swapper/220:15:142
830999105,3cyclictest23964-21grep21:05:262
830999105,3cyclictest18008-21grep19:30:272
830999100,7cyclictest0-21swapper/221:04:012
830899107,2cyclictest0-21swapper/120:30:251
830899107,2cyclictest0-21swapper/119:50:141
830899105,3cyclictest0-21swapper/121:00:261
830899100,8cyclictest0-21swapper/121:05:141
830799107,2cyclictest0-21swapper/020:25:240
830799107,2cyclictest0-21swapper/020:10:120
830799103,6cyclictest0-21swapper/021:02:000
199022100,2sleep30-21swapper/323:56:173
83109996,2cyclictest775-1kworker/u9:223:01:383
83109995,2cyclictest16882-21df20:50:153
83109995,2cyclictest0-21swapper/319:25:193
83109994,3cyclictest5084-21latency20:20:183
83109994,3cyclictest4324-21nscd21:45:083
83109994,3cyclictest31522-21sh22:25:203
83109994,3cyclictest31358-21if_eth020:05:203
83109994,3cyclictest29902-21apt-get21:50:123
83109994,3cyclictest2841-21aten2.4-expect22:30:133
83109994,3cyclictest17464-21sed19:30:183
83109994,3cyclictest13980-21rm22:40:413
83109994,3cyclictest12919-21sed20:40:143
83109994,3cyclictest10826-21cpuspeed_turbos20:35:133
83109991,3cyclictest0-21swapper/320:00:173
83109990,8cyclictest0-21swapper/323:05:203
83109990,6cyclictest0-21swapper/319:23:583
83089994,3cyclictest31385-21interrupts20:05:201
83089991,2cyclictest4339-21munin-node1
83079996,2cyclictest0-21swapper/020:55:170
83079991,6cyclictest0-21swapper/020:21:530
83079980,6cyclictest0-21swapper/020:30:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional