You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-26 - 09:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Sat Apr 26, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
165551995344,5cyclictest196024-21idleruntime-cro20:40:003
165546995238,10cyclictest177979-21munin-plugin-st19:45:012
165551994635,7cyclictest0-21swapper/322:20:013
165546994435,5cyclictest197700-21sh20:45:002
165542994431,8cyclictest204664-21grep21:05:281
165542994431,8cyclictest204664-21grep21:05:271
165551994335,5cyclictest245661-21munin-plugin-st23:30:003
165551994334,6cyclictest214244-21sh21:40:003
165542994232,5cyclictest183055-21munin-plugin-st20:00:011
165551994133,4cyclictest0-21swapper/320:25:003
165542994131,6cyclictest167260-21sh19:15:011
165542994128,10cyclictest231ktimers/119:20:141
165540994127,8cyclictest221483-21grep22:00:270
165540994127,8cyclictest221483-21grep22:00:260
165546994030,7cyclictest167383-21sort19:15:012
165546993932,5cyclictest257114-21cron00:10:002
165542993927,7cyclictest0-21swapper/122:10:001
16554099395,5cyclictest131rcu_preempt22:55:100
165551993831,4cyclictest0-21swapper/321:50:013
165551993826,9cyclictest221894-21auditctl22:05:013
165551993826,9cyclictest221894-21auditctl22:05:003
16554099382,6cyclictest131rcu_preempt21:10:170
165540993826,8cyclictest233465-21latency_hist22:45:010
165540993824,11cyclictest121ktimers/023:20:120
165551993727,7cyclictest188775-21sh20:20:003
165551993727,7cyclictest188775-21sh20:20:003
165542993726,8cyclictest258844-21latency_hist00:15:011
16554099373,5cyclictest131rcu_preempt21:00:140
165540993730,3cyclictest0-21swapper/019:40:020
165540993729,3cyclictest0-21swapper/023:15:010
165540993726,7cyclictest0-21swapper/023:59:590
165540993726,6cyclictest231640-21apt22:40:000
16554099371,6cyclictest131rcu_preempt20:50:080
165551993629,3cyclictest0-21swapper/319:35:183
165551993628,4cyclictest0-21swapper/321:00:173
165546993629,4cyclictest184751-21cron20:05:002
165546993625,6cyclictest254367-21grep23:55:252
165542993628,4cyclictest0-21swapper/122:10:121
165542993626,5cyclictest0-21swapper/119:50:171
165551993528,4cyclictest192612-21latency_hist20:30:013
165551993527,4cyclictest0-21swapper/322:40:283
165551993527,4cyclictest0-21swapper/319:15:173
165542993528,4cyclictest235885-21sh22:55:001
165551993426,4cyclictest0-21swapper/322:55:233
165551993426,4cyclictest0-21swapper/321:20:213
165542993425,5cyclictest0-21swapper/121:50:001
165542993423,7cyclictest0-21swapper/122:55:231
165540993427,4cyclictest0-21swapper/020:20:000
165540993427,4cyclictest0-21swapper/020:20:000
165540993426,4cyclictest194289-21cron20:35:000
165540993422,8cyclictest225443-21grep22:15:220
165551993326,4cyclictest0-21swapper/321:10:003
165551993326,4cyclictest0-21swapper/321:10:003
165551993325,4cyclictest0-21swapper/321:50:213
165551993325,4cyclictest0-21swapper/319:25:273
165551993323,7cyclictest440-21cron20:10:013
165546993325,4cyclictest0-21swapper/220:25:002
165546993321,7cyclictest188375-21grep20:15:292
165546993321,7cyclictest188375-21grep20:15:282
165546993321,7cyclictest0-21swapper/220:30:302
165542993323,6cyclictest0-21swapper/121:20:001
165540993326,3cyclictest0-21swapper/000:20:000
16554099332,4cyclictest131rcu_preempt20:10:100
165540993324,6cyclictest0-21swapper/023:20:000
165540993324,6cyclictest0-21swapper/019:15:020
165540993323,7cyclictest182586-21grep19:55:280
165551993225,3cyclictest0-21swapper/300:30:113
165551993225,3cyclictest0-21swapper/300:30:113
165546993225,3cyclictest0-21swapper/222:00:012
165546993225,3cyclictest0-21swapper/222:00:002
165546993224,4cyclictest0-21swapper/223:50:262
165546993224,4cyclictest0-21swapper/221:35:262
165546993223,6cyclictest260993-21sort00:20:012
165546993222,6cyclictest0-21swapper/222:50:002
165542993222,5cyclictest0-21swapper/123:35:001
165540993224,3cyclictest0-21swapper/023:45:000
165551993125,3cyclictest0-21swapper/322:40:003
165551993123,4cyclictest262164-21grep00:20:283
165546993125,3cyclictest0-21swapper/220:40:012
165546993124,3cyclictest0-21swapper/221:20:002
165546993123,4cyclictest0-21swapper/221:50:122
165546993123,4cyclictest0-21swapper/220:10:222
165542993123,5cyclictest198340-21grep20:50:011
165542993123,4cyclictest0-21swapper/100:00:271
165542993121,6cyclictest195515-21grep20:35:261
165542993113,3cyclictest131rcu_preempt20:18:291
165542993113,3cyclictest131rcu_preempt20:18:281
165540993124,3cyclictest0-21swapper/022:15:000
165540993121,6cyclictest0-21swapper/019:40:290
165551993024,4cyclictest227644-21idleruntime-cro22:25:013
165551993021,6cyclictest0-21swapper/320:10:163
165546993022,4cyclictest0-21swapper/220:00:002
165546993020,6cyclictest0-21swapper/222:00:262
165546993020,6cyclictest0-21swapper/222:00:262
165546993019,7cyclictest250287-21grep23:40:242
165546993019,6cyclictest0-21swapper/222:40:012
165542993022,4cyclictest0-21swapper/123:50:261
165542993021,4cyclictest0-21swapper/120:20:171
165542993020,7cyclictest0-21swapper/119:20:011
165542993011,5cyclictest176807-21if_eth019:40:201
165542993010,5cyclictest131rcu_preempt20:55:191
16555199297,4cyclictest251768-21cut23:50:173
165551992922,4cyclictest0-21swapper/322:00:003
165551992922,4cyclictest0-21swapper/322:00:003
165551992922,3cyclictest0-21swapper/319:55:283
165551992921,5cyclictest0-21swapper/322:10:123
165551992921,4cyclictest260846-21latency_hist00:20:003
165551992921,4cyclictest0-21swapper/322:30:223
165551992920,4cyclictest0-21swapper/321:20:013
165546992919,6cyclictest203426-21cron21:05:002
165546992919,6cyclictest0-21swapper/200:04:592
16554299294,8cyclictest24-21ksoftirqd/123:15:001
16554299293,8cyclictest24-21ksoftirqd/121:15:001
165542992924,3cyclictest231ktimers/120:54:431
165542992921,5cyclictest0-21swapper/100:15:221
165542992921,4cyclictest0-21swapper/121:30:011
165542992920,6cyclictest230406-21tune2fs22:35:141
165542992918,7cyclictest0-21swapper/121:40:261
165540992921,3cyclictest0-21swapper/021:40:010
165540992920,6cyclictest247792-21fschecks_time23:35:180
165540992920,5cyclictest0-21swapper/021:50:000
165551992821,4cyclictest255836-21ls00:00:253
165551992820,5cyclictest0-21swapper/323:35:263
165551992820,4cyclictest0-21swapper/323:55:253
165546992820,4cyclictest192161-21grep20:25:282
165546992819,6cyclictest261867-21fgrep00:20:272
165546992819,4cyclictest0-21swapper/219:30:282
165546992818,6cyclictest0-21swapper/200:35:002
165546992818,6cyclictest0-21swapper/200:35:002
16554299286,7cyclictest24-21ksoftirqd/100:37:321
16554299285,8cyclictest0-21swapper/120:40:271
16554299284,5cyclictest24-21ksoftirqd/119:45:181
165542992822,3cyclictest231ktimers/121:31:521
165542992819,6cyclictest239288-21sh23:05:001
165542992819,6cyclictest231ktimers/120:07:381
165542992819,4cyclictest0-21swapper/122:00:011
165542992819,4cyclictest0-21swapper/122:00:001
165542992817,8cyclictest249058-21sendmail23:40:001
165542992810,10cyclictest131rcu_preempt22:40:111
165540992822,3cyclictest0-21swapper/019:20:270
165540992821,3cyclictest0-21swapper/020:25:000
165540992821,3cyclictest0-21swapper/000:40:000
165540992818,6cyclictest0-21swapper/023:30:000
165540992818,6cyclictest0-21swapper/019:34:030
165551992720,4cyclictest0-21swapper/300:15:013
165551992719,5cyclictest174408-21kworker/u8:1+events_unbound20:05:003
165551992719,5cyclictest0-21swapper/319:10:293
165551992719,4cyclictest0-21swapper/319:40:213
165546992721,3cyclictest0-21swapper/223:05:012
165546992719,4cyclictest0-21swapper/221:05:272
165546992719,4cyclictest0-21swapper/221:05:272
165546992718,6cyclictest0-21swapper/223:20:262
165546992718,6cyclictest0-21swapper/221:25:012
165546992718,4cyclictest0-21swapper/222:30:212
165546992716,6cyclictest258806-21apt-get00:15:012
16554299275,5cyclictest24-21ksoftirqd/123:20:241
16554299273,7cyclictest24-21ksoftirqd/123:20:001
16554299273,6cyclictest0-21swapper/119:30:171
165542992721,4cyclictest231ktimers/123:29:261
165542992719,6cyclictest231ktimers/123:06:571
165542992719,6cyclictest231ktimers/123:06:561
165542992719,4cyclictest0-21swapper/119:25:251
165542992717,7cyclictest213640-21sort21:35:251
165542992716,8cyclictest253935-21ps23:55:231
165542992714,7cyclictest131rcu_preempt22:18:481
165542992710,4cyclictest131rcu_preempt19:38:071
165542992710,4cyclictest131rcu_preempt00:20:021
165540992721,3cyclictest0-21swapper/021:35:010
165540992721,3cyclictest0-21swapper/019:55:000
165540992720,3cyclictest0-21swapper/022:25:010
165540992718,6cyclictest0-21swapper/023:50:260
165551992619,4cyclictest0-21swapper/323:40:243
165551992619,4cyclictest0-21swapper/321:00:013
165551992617,6cyclictest0-21swapper/320:50:213
165551992617,6cyclictest0-21swapper/320:30:313
165546992618,5cyclictest0-21swapper/223:50:002
165546992617,6cyclictest0-21swapper/221:15:002
165546992616,7cyclictest233460-21date22:45:012
165546992615,8cyclictest240670-21awk23:10:152
16554299268,3cyclictest131rcu_preempt22:30:071
16554299265,6cyclictest24-21ksoftirqd/121:55:011
165542992619,4cyclictest228258-21sh22:30:001
165542992618,6cyclictest199746-21kworker/u8:1+events_unbound21:04:511
165542992617,3cyclictest131rcu_preempt20:12:161
165542992616,4cyclictest131rcu_preempt20:01:191
165542992614,5cyclictest131rcu_preempt00:05:561
165542992613,3cyclictest131rcu_preempt00:34:111
165542992613,3cyclictest131rcu_preempt00:34:111
165542992610,9cyclictest131rcu_preempt20:28:091
165540992617,5cyclictest0-21swapper/020:45:000
165540992614,4cyclictest191271-1kworker/0:2H+mmc_complete21:00:010
165540992610,4cyclictest151rcuc/019:25:240
165551992518,4cyclictest0-21swapper/323:00:173
165551992518,4cyclictest0-21swapper/322:10:003
165546992519,3cyclictest0-21swapper/221:30:132
165546992518,3cyclictest0-21swapper/222:10:192
165546992517,5cyclictest0-21swapper/219:15:022
165546992515,6cyclictest199601-21grep20:50:212
16554299258,9cyclictest131rcu_preempt21:20:201
165542992518,4cyclictest231ktimers/100:30:001
165542992518,4cyclictest0-21swapper/122:00:261
165542992518,4cyclictest0-21swapper/122:00:261
165542992516,6cyclictest0-21swapper/120:35:001
165542992514,4cyclictest131rcu_preempt22:21:191
165540992518,4cyclictest192196-21head20:25:290
165540992518,3cyclictest0-21swapper/000:15:010
165540992516,6cyclictest196002-21idleruntime-cro20:40:000
16554099251,3cyclictest131rcu_preempt21:55:190
16554099251,3cyclictest131rcu_preempt21:55:190
16554099251,20cyclictest0-21swapper/021:25:310
16554099251,20cyclictest0-21swapper/021:06:330
16554099251,20cyclictest0-21swapper/021:06:330
165551992418,4cyclictest203325-21kworker/u8:0+events_unbound21:12:023
165551992418,3cyclictest0-21swapper/322:50:193
16555199241,6cyclictest173335-21perl19:30:193
165551992416,5cyclictest0-21swapper/321:40:273
165551992413,8cyclictest169833-21diskstats19:20:163
165546992419,3cyclictest245752-21latency_hist23:30:012
165546992419,3cyclictest0-21swapper/221:45:002
165546992418,4cyclictest242917-21unixbench_singl23:15:282
165546992417,4cyclictest0-21swapper/222:50:272
165546992416,4cyclictest0-21swapper/223:35:272
165542992417,4cyclictest250335-21unixbench_singl23:40:241
16554299241,6cyclictest24-21ksoftirqd/122:45:051
16554299241,17cyclictest0-21swapper/123:47:051
165540992419,3cyclictest0-21swapper/000:25:000
165540992416,3cyclictest0-21swapper/021:40:270
165540992414,7cyclictest185562-21sed20:10:010
165540992413,8cyclictest263626-21awk00:30:120
165540992413,8cyclictest263626-21awk00:30:110
165540992412,9cyclictest257163-21sed00:10:020
16554099241,19cyclictest0-21swapper/023:46:550
16554099241,19cyclictest0-21swapper/023:07:030
16554099241,19cyclictest0-21swapper/023:07:030
16554099241,19cyclictest0-21swapper/022:26:400
16554099241,19cyclictest0-21swapper/021:19:560
16554099241,19cyclictest0-21swapper/000:27:120
16554099240,20cyclictest0-21swapper/020:45:260
16555199236,4cyclictest1803822missed_timers19:50:213
165551992317,4cyclictest258449-21kworker/u8:3+events_unbound00:29:003
165551992316,4cyclictest0-21swapper/322:30:003
165551992315,5cyclictest391ktimers/320:40:253
165551992313,6cyclictest242856-21unixbench_multi23:15:293
16554699233,8cyclictest178198-21/usr/sbin/munin19:45:242
165546992317,3cyclictest0-21swapper/223:00:002
165546992317,3cyclictest0-21swapper/222:20:222
165546992314,6cyclictest185428-21cat20:10:002
165540992317,3cyclictest0-21swapper/019:15:270
165540992315,6cyclictest227279-1kworker/0:0H+mmc_complete22:50:180
165540992315,4cyclictest0-21swapper/020:00:190
165540992313,7cyclictest247425-21date23:35:010
165540992312,7cyclictest255235-21cut00:00:170
16554099231,18cyclictest0-21swapper/022:47:450
16554099231,18cyclictest0-21swapper/022:06:410
16554099230,21cyclictest0-21swapper/019:46:060
16554099230,19cyclictest0-21swapper/021:52:040
16554099230,19cyclictest0-21swapper/021:21:310
165551992216,4cyclictest208178-21kworker/u8:1+events_unbound21:27:283
165551992215,5cyclictest0-21swapper/323:30:213
165551992215,3cyclictest0-21swapper/321:30:213
165551992215,3cyclictest0-21swapper/319:50:013
165551992212,7cyclictest234204-21basename22:50:003
16554699224,15cyclictest225447-21unixbench_multi22:15:222
165546992216,3cyclictest0-21swapper/223:30:102
165546992216,3cyclictest0-21swapper/220:55:172
165546992215,3cyclictest0-21swapper/200:35:162
165540992215,3cyclictest0-21swapper/023:03:170
165540992212,6cyclictest228618-21grep22:30:110
165551992113,5cyclictest0-21swapper/300:35:173
16554699217,4cyclictest215610-21kworker/u8:2+events_unbound21:49:012
165546992113,6cyclictest0-21swapper/219:20:222
165546992112,6cyclictest174897-21expr19:35:152
165546992111,7cyclictest263275-21awk00:30:002
165551992013,5cyclictest391ktimers/323:20:123
165546992014,3cyclictest0-21swapper/219:25:252
165546992013,4cyclictest0-21swapper/221:30:022
16554699198,8cyclictest198029-21seq20:46:232
165546991913,3cyclictest228260-21kthreadd22:30:002
165546991913,3cyclictest222189-21sh22:06:002
165546991912,4cyclictest0-21swapper/219:50:262
16555199181,10cyclictest637-21runrttasks23:05:103
16555199181,10cyclictest637-21runrttasks23:05:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional