You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 21:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Apr 07, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15345991262,86cyclictest15339-21cyclictest16:30:420
2656121250,6sleep31534899cyclictest19:08:183
1143021190,3sleep011409-21ssh18:50:560
15345991091,4cyclictest0-21swapper/020:51:590
15345991031,92cyclictest19020-21tr16:40:520
15345991031,92cyclictest19020-21tr16:40:520
15345991031,91cyclictest24700-21ssh20:21:090
15345991011,90cyclictest8468-21apt21:20:390
1534599988,63cyclictest21535-21proc_pri16:46:040
5392920,5sleep31534899cyclictest21:10:533
1534599923,70cyclictest18801-21idleruntime-cro20:15:390
1534599922,76cyclictest0-21swapper/017:41:090
1534599921,73cyclictest22689-21latency_hist18:20:400
1534599921,72cyclictest0-21swapper/019:06:070
1534599911,2cyclictest0-21swapper/019:50:400
1534599911,2cyclictest0-21swapper/019:50:400
1534599892,67cyclictest0-21swapper/020:41:010
1534599881,70cyclictest17504-21tune2fs16:35:570
1534599881,4cyclictest14124-21unixbench-2d19:31:070
1534599871,48cyclictest2741-21latency_hist17:25:410
1534599859,55cyclictest3556-21latency_hist20:35:400
1534599859,55cyclictest3556-21latency_hist20:35:400
1534599851,67cyclictest25885-21grep21:01:080
1534599851,66cyclictest30726-21ssh21:46:060
1534599851,4cyclictest19443-21fschecks_count20:15:580
1534599832,69cyclictest0-21swapper/018:25:520
1534599832,69cyclictest0-21swapper/018:25:520
15345998314,34cyclictest27447-21cat18:31:060
1534599822,68cyclictest0-21swapper/019:16:000
1534599821,66cyclictest4411-21latency_hist17:30:400
1534599816,59cyclictest0-21swapper/021:05:530
1534599814,63cyclictest0-21swapper/020:00:590
1534599814,63cyclictest0-21swapper/020:00:590
1534599811,67cyclictest31021-21grep19:51:030
1534599802,61cyclictest0-21swapper/017:35:540
1534599792,61cyclictest0-21swapper/021:12:090
1534599791,4cyclictest908-21runrttasks19:25:340
1534599791,3cyclictest19698-21ssh19:00:500
1534599782,63cyclictest0-21swapper/021:51:410
1534599781,64cyclictest26114-21diskmemload21:26:020
1534599781,64cyclictest0-21swapper/019:56:100
112162780,6sleep11534699cyclictest20:06:011
1534599771,63cyclictest0-21swapper/018:36:310
1534599771,62cyclictest0-21swapper/019:25:510
1534599762,63cyclictest0-21swapper/021:37:370
1534599762,63cyclictest0-21swapper/021:37:370
1534599762,63cyclictest0-21swapper/017:01:520
1534599762,56cyclictest0-21swapper/017:30:570
1534599762,56cyclictest0-21swapper/017:30:570
1534599761,63cyclictest0-21swapper/019:14:220
1534599761,5cyclictest17268-21grep19:35:540
1534599761,4cyclictest9415-21ls21:21:000
1534599761,4cyclictest9415-21ls21:21:000
1534599761,4cyclictest12604-21ssh20:07:220
1534599761,2cyclictest0-21swapper/021:40:460
1534599761,2cyclictest0-21swapper/019:45:050
1534599760,3cyclictest20543-21ssh21:34:490
51482750,2sleep20-21swapper/218:43:172
1534599757,42cyclictest0-21swapper/018:10:530
56062740,2sleep30-21swapper/321:55:463
1534599741,60cyclictest0-21swapper/018:55:440
1534599741,60cyclictest0-21swapper/018:55:440
1534599741,5cyclictest28381-21cat20:26:000
1534599741,47cyclictest0-21swapper/018:47:190
1534599741,37cyclictest13929-21chrt20:46:590
1534599731,4cyclictest917-21snmpd17:49:020
1534599731,2cyclictest0-21swapper/018:20:580
1534699722,15cyclictest7750irq/56-dwc_otg17:40:511
1534599722,63cyclictest0-21swapper/018:10:070
1534599722,57cyclictest0-21swapper/017:57:170
1534599721,2cyclictest0-21swapper/018:42:130
1534599712,32cyclictest4198-21fschecks_time20:35:580
1534599711,33cyclictest0-21swapper/017:15:170
1534599701,4cyclictest7820-21ssh21:57:400
178312690,2sleep30-21swapper/320:14:013
1534599691,3cyclictest0-21swapper/016:59:190
1534599691,2cyclictest0-21swapper/020:58:170
268972680,2sleep20-21swapper/219:08:532
1534599681,50cyclictest12617-21ntpq17:51:020
1534599681,3cyclictest0-21swapper/018:01:520
15345996810,34cyclictest0-21swapper/016:51:040
194292650,2sleep20-21swapper/219:00:402
194292650,2sleep20-21swapper/219:00:402
1534599642,59cyclictest0-21swapper/017:17:580
1534599641,2cyclictest0-21swapper/017:07:310
312432630,1sleep10-21swapper/121:46:221
213242560,2sleep30-21swapper/320:17:393
15348995518,20cyclictest24443-21ls20:21:073
15348995514,20cyclictest22736-21/usr/sbin/munin18:20:593
260772540,2sleep30-21swapper/321:01:233
1534799546,29cyclictest0-21swapper/219:40:532
1534699544,6cyclictest7750irq/56-dwc_otg19:25:521
240892530,2sleep10-21swapper/121:00:201
15347995319,19cyclictest29862-21ssh19:11:352
15346995123,6cyclictest20116-21cron20:55:391
15348995015,25cyclictest16073-21ssh21:29:143
1534799506,24cyclictest7950irq/56-dwc_otg_20:49:562
15347995013,20cyclictest5128-21ls17:30:592
15347995013,20cyclictest5128-21ls17:30:592
15348994912,12cyclictest4633-21diskstats19:20:553
15348994910,24cyclictest31239-21sort19:51:063
15348994910,23cyclictest27986-21ssh18:31:303
15347994916,18cyclictest30302-21ssh21:06:322
1534699492,5cyclictest0-21swapper/117:50:501
15347994818,11cyclictest13960-21/usr/sbin/munin19:31:052
15347994812,18cyclictest0-21swapper/221:40:522
15347994811,6cyclictest0-21swapper/221:00:512
1534899478,25cyclictest0-21swapper/319:30:523
15348994710,11cyclictest31482-21ssh19:14:253
1534799479,21cyclictest4135-21sshd21:15:212
1534799477,25cyclictest0-21swapper/216:45:532
15347994713,21cyclictest31090-21ssh20:29:442
1534699472,7cyclictest0-21swapper/119:35:511
1534899467,25cyclictest0-21swapper/317:45:443
15348994622,8cyclictest15373-21apt-get16:30:503
15348994613,11cyclictest19046-21ssh19:37:023
1534799468,20cyclictest19831-21ssh20:55:092
1534799467,26cyclictest0-21swapper/217:55:532
15347994616,22cyclictest8705-21aten_rbpower_cu19:25:532
135142467,25sleep20-21swapper/216:25:492
1534899459,22cyclictest28079-21ssh19:47:493
1534899459,22cyclictest28079-21ssh19:47:493
1534899455,26cyclictest0-21swapper/318:35:533
1534899455,25cyclictest0-21swapper/321:40:453
15348994514,20cyclictest0-21swapper/319:28:053
15348994513,15cyclictest381ktimersoftd/319:45:273
15348994513,13cyclictest20049-21ssh20:55:323
15348994513,11cyclictest9530-21netstat21:21:013
15348994513,11cyclictest9530-21netstat21:21:013
1534799455,8cyclictest23375-21ls18:21:002
15347994513,17cyclictest301ktimersoftd/219:47:502
15347994513,17cyclictest301ktimersoftd/219:47:502
15347994512,19cyclictest4424-21latency_hist17:30:402
15347994512,19cyclictest301ktimersoftd/220:59:212
15347994511,18cyclictest24776-21ssh20:21:122
15347994511,16cyclictest4666-21sh21:54:292
148452453,10sleep00-21swapper/016:27:490
1534899449,24cyclictest18108-21sed16:36:063
1534899448,20cyclictest0-21swapper/320:27:153
15348994417,12cyclictest8550-21latency_hist21:20:403
15348994414,12cyclictest19696-21apt-get19:00:503
15348994414,11cyclictest9176-21/usr/sbin/munin17:41:083
1534799449,14cyclictest24794-21tune2fs18:25:552
1534799449,14cyclictest24794-21tune2fs18:25:552
1534799448,21cyclictest19700-21/usr/sbin/munin20:16:012
1534799448,13cyclictest25062-21ssh21:40:262
1534799448,13cyclictest25062-21ssh21:40:262
1534799448,12cyclictest8715-21ssh20:41:012
15347994414,23cyclictest15349-21taskset16:30:422
15347994413,12cyclictest19662-21ssh19:00:482
1534799441,28cyclictest18489-21unixbench_singl19:36:082
15347994410,17cyclictest29340-21ssh18:33:502
15346994413,6cyclictest7850irq/56-dwc_otg_19:00:501
51282430,2sleep20-21swapper/219:21:012
15348994318,13cyclictest2455950ssh21:39:313
15348994318,13cyclictest2455950ssh21:39:313
15348994314,14cyclictest11035-21/usr/sbin/munin20:05:593
15348994311,7cyclictest6597-21df_inode17:35:563
15348994311,23cyclictest7750irq/56-dwc_otg18:47:193
15348994311,12cyclictest31049-21ssh21:46:093
15348994311,11cyclictest1482-21/usr/sbin/munin21:50:503
1534799439,17cyclictest20893-21latency_hist18:15:402
1534799437,18cyclictest3865-21ssh19:20:222
1534799437,12cyclictest30656-21/usr/sbin/munin21:46:052
1534799436,25cyclictest0-21swapper/221:30:532
15347994328,8cyclictest7750irq/56-dwc_otg21:26:202
15347994315,11cyclictest301ktimersoftd/220:15:392
15347994313,16cyclictest7750irq/56-dwc_otg18:47:192
15346994311,9cyclictest23-21ksoftirqd/117:35:511
78842420,3sleep123-21ksoftirqd/118:46:061
77972420,6sleep11534699cyclictest20:40:401
1534899429,18cyclictest7950irq/56-dwc_otg_20:35:493
1534899429,15cyclictest2009-21ssh19:17:083
1534899429,12cyclictest5246-21grep17:31:013
1534899429,12cyclictest5246-21grep17:31:013
1534899425,14cyclictest7950irq/56-dwc_otg_20:41:063
15348994211,17cyclictest0-21swapper/316:46:063
1534799429,12cyclictest9011-21expr17:41:062
1534799428,22cyclictest0-21swapper/216:50:532
1534799426,23cyclictest0-21swapper/216:40:522
1534799426,23cyclictest0-21swapper/216:40:522
1534699427,22cyclictest0-21swapper/118:00:521
15346994220,4cyclictest22248-21ntp_kernel_err19:41:011
1534899416,25cyclictest0-21swapper/317:00:533
1534899416,23cyclictest0-21swapper/317:55:533
1534799417,25cyclictest3560-21ssh19:56:312
1534799413,12cyclictest7850irq/56-dwc_otg_16:36:042
1534799412,11cyclictest11446-21ssh21:23:342
1534799412,11cyclictest11446-21ssh21:23:342
15347994114,9cyclictest3317-21sh20:35:242
15347994114,9cyclictest3317-21sh20:35:242
15347994114,6cyclictest7850irq/56-dwc_otg_18:20:392
1534799411,26cyclictest7950irq/56-dwc_otg_20:08:082
1534699414,25cyclictest7850irq/56-dwc_otg_17:56:041
15346994121,10cyclictest7850irq/56-dwc_otg_21:40:411
15346994110,9cyclictest31958-21ssh20:30:491
15346994110,9cyclictest31958-21ssh20:30:491
1534899407,20cyclictest0-21swapper/316:50:533
15348994013,7cyclictest19198-21df_inode16:40:543
15348994013,7cyclictest19198-21df_inode16:40:543
15348994013,11cyclictest24944-21if_eth018:25:563
15348994013,11cyclictest24944-21if_eth018:25:563
15348994013,10cyclictest20771-21ssh21:35:133
15348994010,11cyclictest8212-21ssh20:02:103
15348994010,11cyclictest8212-21ssh20:02:103
15348994010,11cyclictest30506-21ssh21:06:543
1534799405,12cyclictest32506-21ssh18:36:542
15347994013,11cyclictest7950irq/56-dwc_otg_19:50:432
1534699406,5cyclictest5527-21ssh21:55:411
1534699405,13cyclictest3061-21timerandwakeup19:56:081
1534899398,13cyclictest15491-21ssh20:49:473
1534899394,21cyclictest7950irq/56-dwc_otg_20:32:593
1534899394,21cyclictest7950irq/56-dwc_otg_20:32:593
15348993916,5cyclictest2374-21fschecks_time19:55:583
15348993915,15cyclictest381ktimersoftd/320:58:023
15348993911,12cyclictest15374-21seq18:55:473
15348993911,12cyclictest15374-21seq18:55:473
1534799396,14cyclictest7950irq/56-dwc_otg_21:57:462
1534799395,22cyclictest8017-21ssh20:01:492
1534799395,22cyclictest8017-21ssh20:01:492
15347993915,6cyclictest301ktimersoftd/221:20:402
15346993919,4cyclictest27498-21grep20:25:401
15346993910,23cyclictest7950irq/56-dwc_otg_21:05:511
1534899388,16cyclictest0-21swapper/318:45:003
1534899388,11cyclictest13547-21ssh18:53:013
1534899384,10cyclictest0-21swapper/318:20:403
1534899383,23cyclictest0-21swapper/317:28:253
1534799387,24cyclictest0-21swapper/217:50:512
15347993820,4cyclictest15798-21fschecks_count18:00:552
15347993818,12cyclictest7850irq/56-dwc_otg_18:55:252
15347993813,11cyclictest301ktimersoftd/220:35:412
1534699388,15cyclictest7950irq/56-dwc_otg_18:30:531
15346993818,5cyclictest2451050wget16:55:531
15346993814,6cyclictest7950irq/56-dwc_otg_16:35:531
15346993811,4cyclictest0-21swapper/121:00:551
1534899377,20cyclictest0-21swapper/318:13:053
1534799379,12cyclictest7290-21date17:36:062
15347993717,10cyclictest7850irq/56-dwc_otg_17:25:282
1534699377,8cyclictest0-21swapper/121:35:541
1534699377,8cyclictest0-21swapper/121:35:541
1534699374,6cyclictest11817-21awk20:45:391
1534699371,27cyclictest31153-21switchtime19:51:051
1534699371,26cyclictest21875-21timerandwakeup18:16:051
1534699371,24cyclictest16124-21ls20:50:401
15346993711,14cyclictest0-21swapper/118:26:061
15346993711,14cyclictest0-21swapper/118:26:061
1534899368,4cyclictest0-21swapper/318:01:063
1534699361,7cyclictest15790-21ssh20:11:041
15346993617,4cyclictest29713-21sh19:50:401
15346993617,4cyclictest29713-21sh19:50:401
1534699361,16cyclictest0-21swapper/117:06:081
1534899354,5cyclictest0-21swapper/316:56:063
1534899353,18cyclictest7950irq/56-dwc_otg_18:06:063
1534899353,11cyclictest0-21swapper/317:12:273
1534799351,22cyclictest0-21swapper/216:55:522
1534699358,17cyclictest0-21swapper/121:18:021
1534699357,13cyclictest7850irq/56-dwc_otg_16:41:061
1534699357,13cyclictest7850irq/56-dwc_otg_16:41:061
1534699351,7cyclictest917-21snmpd20:29:361
15346993512,5cyclictest0-21swapper/121:56:311
1534699351,24cyclictest23262-21latency18:20:591
1534699351,20cyclictest10372-21cut17:45:561
1534899346,14cyclictest0-21swapper/317:50:513
1534799347,14cyclictest0-21swapper/217:05:542
1534799345,10cyclictest27474-21chrt17:02:172
1534699343,7cyclictest0-21swapper/119:30:501
1534699343,5cyclictest0-21swapper/121:25:401
1534699343,5cyclictest0-21swapper/121:25:401
1534699342,7cyclictest7850irq/56-dwc_otg_16:30:421
1534699342,17cyclictest7950irq/56-dwc_otg_17:20:571
15346993417,5cyclictest19600-21chrt21:33:061
1534699341,22cyclictest24217-21ls16:55:391
150722346,18sleep315107-21munin-run16:30:403
1534899336,9cyclictest7950irq/56-dwc_otg_17:20:173
1534899332,6cyclictest1075-21aten_rbpower_vo17:20:543
1534799337,7cyclictest9767-21apt-get17:45:412
1534799336,8cyclictest7750irq/56-dwc_otg17:16:012
1534799335,18cyclictest0-21swapper/218:05:542
1534799331,7cyclictest0-21swapper/217:10:512
1534699339,14cyclictest7750irq/56-dwc_otg17:05:401
1534699338,4cyclictest7750irq/56-dwc_otg19:15:531
1534699336,17cyclictest0-21swapper/121:25:521
1534699335,6cyclictest7750irq/56-dwc_otg17:30:531
1534699335,6cyclictest7750irq/56-dwc_otg17:30:531
1534699332,3cyclictest0-21swapper/118:05:521
15346993315,13cyclictest0-21swapper/119:05:511
15346993312,4cyclictest5022-21memory19:21:001
15346993310,5cyclictest7750irq/56-dwc_otg20:00:531
15346993310,5cyclictest7750irq/56-dwc_otg20:00:531
1534699328,4cyclictest0-21swapper/118:41:061
1534699327,21cyclictest0-21swapper/118:50:521
1534699327,15cyclictest0-21swapper/120:15:541
1534699326,14cyclictest0-21swapper/117:15:511
15346993223,6cyclictest20701-21idleruntime-cro18:15:391
1534699322,12cyclictest0-21swapper/116:46:051
1534699321,21cyclictest29397-21timerwakeupswit19:11:061
15346993211,4cyclictest0-21swapper/121:11:331
1534699312,6cyclictest19575-21latency_hist19:00:411
1534699311,19cyclictest3769-21ls17:26:071
1534699311,18cyclictest31890-21ssh18:36:071
1534899307,13cyclictest0-21swapper/317:05:543
1534699305,4cyclictest0-21swapper/118:56:021
15346993018,6cyclictest30047-21fschecks_count17:10:561
147632286,14sleep10-21swapper/116:26:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional