You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 03:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Apr 11, 2026 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
363399522,30cyclictest171-21systemd-network21:50:122
362399467,29cyclictest24768-21ssh22:05:111
362399461,28cyclictest9888-21sh23:07:411
362399460,41cyclictest8090-21apt-get20:55:131
362399452,39cyclictest0-21swapper/121:45:131
3623994418,17cyclictest21-21ksoftirqd/100:10:131
361599431,32cyclictest26676-21ssh23:00:130
362399420,37cyclictest12222-21apt-get23:10:141
362399420,37cyclictest12222-21apt-get23:10:131
363399414,29cyclictest27-21ksoftirqd/220:55:122
362399412,34cyclictest6639-21apt-get20:20:121
362399411,35cyclictest0-21swapper/121:55:121
362399401,35cyclictest21176-21apt-get23:15:121
362399401,32cyclictest29309-21apt-get20:10:121
362399392,30cyclictest301-21apt-get19:40:131
362399391,34cyclictest17746-21apt-get21:05:121
362399390,36cyclictest0-21swapper/121:50:131
362399390,35cyclictest5190-21apt-get19:45:141
3647993820,12cyclictest28366irq/58-eth023:35:143
363399382,18cyclictest21599-21ssh22:38:422
362399382,33cyclictest0-21swapper/121:40:141
362399382,33cyclictest0-21swapper/120:00:121
362399381,33cyclictest0-21swapper/100:05:151
362399380,34cyclictest26675-21apt-get23:00:131
3623993720,12cyclictest0-21swapper/100:30:121
362399371,33cyclictest0-21swapper/121:15:121
362399371,31cyclictest10321-21ssh23:45:131
363399364,21cyclictest15828-21ssh23:30:142
362399362,29cyclictest0-21swapper/122:10:121
362399361,32cyclictest5094-21apt-get22:30:131
362399361,32cyclictest30251-21apt-get23:20:121
362399361,30cyclictest0-21swapper/121:20:151
362399360,33cyclictest0-21swapper/121:35:111
3615993621,8cyclictest15808-21apt-get22:00:120
3615993620,12cyclictest0-21swapper/000:30:110
362399353,28cyclictest0-21swapper/123:40:121
362399353,28cyclictest0-21swapper/123:40:111
362399351,30cyclictest11505-21apt-get20:25:131
362399351,29cyclictest0-21swapper/119:25:131
362399350,33cyclictest23104-21apt-get22:40:141
362399350,31cyclictest26465-21apt-get21:30:151
362399350,30cyclictest6823-21apt-get23:25:131
362399350,23cyclictest24823-21ssh21:28:321
362399350,20cyclictest22330-21diskmemload21:12:141
3615993522,8cyclictest8319-21apt-get00:20:110
363399342,22cyclictest30254-21ssh23:20:112
362399341,31cyclictest0-21swapper/122:00:131
362399341,30cyclictest0-21swapper/123:35:111
362399340,19cyclictest0-21swapper/123:54:051
363399334,3cyclictest131rcu_preempt00:15:132
362399331,29cyclictest19291-21latency23:30:231
362399331,29cyclictest17659-21ssh22:55:101
362399331,27cyclictest21165-21apt-get20:35:121
362399331,25cyclictest4913-21apt-get00:00:141
362399330,29cyclictest18028-21sshd19:23:391
362399330,29cyclictest1726-21apt-get20:15:141
3623993210,19cyclictest14660-21irqrtprio22:35:171
362399320,29cyclictest0-21swapper/123:55:131
362399320,28cyclictest0-21swapper/122:25:131
362399320,28cyclictest0-21swapper/121:00:131
364799316,13cyclictest28366irq/58-eth021:40:143
3647993112,13cyclictest28366irq/58-eth023:45:143
3633993110,15cyclictest27-21ksoftirqd/223:10:132
3633993110,15cyclictest27-21ksoftirqd/223:10:132
362399311,27cyclictest21274-21ssh22:20:211
362399311,27cyclictest17556-21fschecks_time00:25:161
362399311,27cyclictest12090-21ntp_states19:15:261
362399311,27cyclictest11996-21ntp_192.168.11400:20:251
362399311,27cyclictest10093-21diskstats19:50:151
362399311,26cyclictest30849-21apt-get20:45:121
362399310,28cyclictest24500-21apt-get20:05:131
362399310,20cyclictest22330-21diskmemload00:39:411
361599313,23cyclictest22809-21ssh00:10:150
362399301,26cyclictest18151-21memory19:55:221
362399301,25cyclictest12345-21/usr/sbin/munin22:50:251
362399301,25cyclictest0-21swapper/120:40:131
362399300,26cyclictest31737-21apt-get00:15:141
362399300,26cyclictest0-21swapper/122:45:131
361599306,13cyclictest24843-21ssh23:35:120
361599306,10cyclictest3653-21latency19:40:220
361599305,10cyclictest21991-21ssh23:32:030
3615993025,3cyclictest18269-21apt-get19:25:120
3615993025,2cyclictest13825-21apt-get00:05:140
361599301,22cyclictest14111-21ssh22:35:140
363399294,16cyclictest22346irq/40-dwc2_hso22:00:122
363399293,17cyclictest22346irq/40-dwc2_hso20:20:132
363399293,11cyclictest0-21swapper/222:11:402
363399292,20cyclictest0-21swapper/219:45:142
363399292,19cyclictest531-21usb-storage23:05:122
363399292,11cyclictest12660-21ssh23:26:422
363399291,10cyclictest4916-21ssh00:00:142
362399297,19cyclictest22330-21diskmemload22:15:081
362399292,24cyclictest0-21swapper/119:10:101
362399291,25cyclictest28021-21df19:35:151
362399291,25cyclictest19671-21memory20:30:231
362399291,24cyclictest6685-21ntp_192.168.11420:50:231
362399291,23cyclictest27511-21usage_EthcTask19:30:271
361599295,8cyclictest27909-21ntp_192.168.11420:05:230
361599295,10cyclictest4978-21ssh00:16:260
361599294,7cyclictest0-21swapper/020:40:150
361599294,22cyclictest12105-21ssh21:40:120
3615992922,4cyclictest24763-21ssh22:05:120
3615992921,5cyclictest14836-21apt-get19:55:130
3615992920,6cyclictest0-21swapper/000:00:150
3615992918,5cyclictest6729-21apt-get21:55:130
364799281,12cyclictest28366irq/58-eth021:25:133
363399283,18cyclictest22346irq/40-dwc2_hso00:05:142
363399282,19cyclictest0-21swapper/219:35:142
361599286,8cyclictest23393-21ssh21:26:510
361599285,10cyclictest13738-21kthreadcore21:00:180
364799279,11cyclictest28366irq/58-eth023:30:143
364799273,17cyclictest0-21swapper/322:38:423
364799272,18cyclictest0-21swapper/322:14:403
363399273,18cyclictest0-21swapper/221:33:402
3633992717,5cyclictest27-21ksoftirqd/222:15:152
363399271,10cyclictest22720-21ssh21:10:112
361599276,7cyclictest2100-21cut20:15:180
361599274,7cyclictest22330-21diskmemload22:17:160
361599272,19cyclictest11699-21sshd22:32:350
3615992721,4cyclictest10312-21apt-get23:45:120
3615992720,3cyclictest3779-21apt-get19:10:110
3615992720,3cyclictest28272-21apt-get23:55:120
3615992719,4cyclictest3247-21apt-get20:50:110
3615992718,5cyclictest31755-21apt-get21:15:130
363399263,15cyclictest22346irq/40-dwc2_hso20:15:112
363399263,14cyclictest22346irq/40-dwc2_hso19:10:122
3633992612,11cyclictest23258-21sshd21:26:412
3633992611,9cyclictest27-21ksoftirqd/220:25:122
363399261,11cyclictest17550-21cut00:25:172
361599269,13cyclictest30146-21ssh21:50:120
361599263,8cyclictest22330-21diskmemload22:47:010
361599263,8cyclictest0-21swapper/022:40:250
361599262,8cyclictest32119-21ssh22:25:260
361599262,19cyclictest24670-21ssh00:28:320
361599262,14cyclictest0-21swapper/020:25:280
3615992613,11cyclictest3011-21apt-get21:35:130
361599261,19cyclictest13470-21ssh23:27:410
364799252,17cyclictest0-21swapper/321:31:403
3647992510,8cyclictest0-21swapper/323:05:123
363399253,14cyclictest22346irq/40-dwc2_hso19:40:132
363399252,8cyclictest26926-21ssh23:53:522
363399252,17cyclictest1385-21ssh23:40:132
363399252,17cyclictest1385-21ssh23:40:122
363399251,9cyclictest10942-21ssh22:31:402
363399251,17cyclictest28263-21find23:55:112
361599252,7cyclictest0-21swapper/022:10:370
361599252,19cyclictest21203-21ssh23:15:130
361599251,9cyclictest17181-21ssh23:10:380
361599251,9cyclictest17181-21ssh23:10:370
361599251,10cyclictest29470-21ssh21:12:480
363399242,17cyclictest29942-21ssh00:30:222
363399242,16cyclictest2842-21ssh00:35:142
363399242,16cyclictest22346irq/40-dwc2_hso21:05:132
363399241,4cyclictest0-21swapper/223:45:132
3633992412,9cyclictest6975-21ssh22:48:252
3633992412,5cyclictest23037-21ssh22:56:052
361599247,10cyclictest0-21swapper/021:45:130
361599244,7cyclictest22330-21diskmemload23:06:200
361599243,8cyclictest0-21swapper/021:20:020
361599243,7cyclictest0-21swapper/023:40:150
361599243,7cyclictest0-21swapper/023:40:140
3615992419,3cyclictest8693-21apt-get22:50:120
3615992414,7cyclictest19498-21ssh22:20:120
3615992414,6cyclictest0-21swapper/019:50:120
364799237,13cyclictest0-21swapper/300:15:123
364799233,13cyclictest0-21swapper/322:32:353
364799232,14cyclictest28366irq/58-eth021:20:143
3647992310,11cyclictest0-21swapper/322:05:123
363399232,4cyclictest0-21swapper/219:30:112
363399232,16cyclictest21174-21ssh23:15:112
363399232,13cyclictest0-21swapper/219:25:122
363399231,7cyclictest29225-21kernelversion22:25:212
363399231,5cyclictest17021-21ls00:25:022
363399231,16cyclictest0-21swapper/220:35:112
361599239,10cyclictest0-21swapper/021:05:130
361599238,12cyclictest0-21swapper/019:20:110
361599235,6cyclictest22330-21diskmemload23:53:070
361599232,11cyclictest12-21ksoftirqd/021:30:520
361599231,9cyclictest13177-21/usr/sbin/munin19:15:310
3615992316,3cyclictest29771-21apt-get23:20:020
3615992310,9cyclictest19663-21apt-get20:00:130
3615992310,8cyclictest0-21swapper/020:55:120
363399223,9cyclictest23674-21kthreadcore00:10:212
363399223,14cyclictest22346irq/40-dwc2_hso20:45:122
363399223,12cyclictest0-21swapper/220:50:112
363399223,10cyclictest7147-21cut21:55:182
363399222,16cyclictest0-21swapper/223:02:412
363399222,11cyclictest27927-21kthreadcore23:35:222
363399222,10cyclictest924-21snmpd20:10:142
362399221,18cyclictest3726-21grep19:10:021
361599227,3cyclictest22330-21diskmemload22:55:190
361599226,11cyclictest0-21swapper/020:10:120
361599225,14cyclictest791-21kthreadcore20:45:210
361599224,7cyclictest0-21swapper/020:35:170
361599223,11cyclictest12-21ksoftirqd/000:35:020
364799213,8cyclictest0-21swapper/319:50:123
364799212,10cyclictest0-21swapper/322:20:133
364799211,16cyclictest28366irq/58-eth023:25:143
364799210,17cyclictest0-21swapper/323:40:113
364799210,17cyclictest0-21swapper/323:40:103
363399219,9cyclictest171-21systemd-network21:22:212
363399214,3cyclictest131rcu_preempt21:40:282
363399213,12cyclictest22346irq/40-dwc2_hso21:35:122
363399212,10cyclictest19679-21fschecks_time22:20:162
363399211,7cyclictest0-21swapper/219:20:132
3633992115,3cyclictest8865-21ssh22:50:142
3633992115,2cyclictest0-21swapper/222:40:122
363399211,17cyclictest20005-21wc20:30:282
363399211,15cyclictest0-21swapper/221:15:142
363399211,10cyclictest131rcu_preempt20:05:202
363399211,10cyclictest10320-21irqcore19:50:182
361599218,9cyclictest28226-21/usr/sbin/munin19:35:180
361599218,10cyclictest19153-21grep20:30:210
361599216,11cyclictest5478-21if_eth119:45:180
364799209,8cyclictest0-21swapper/323:20:083
364799209,6cyclictest0-21swapper/321:14:013
364799208,9cyclictest0-21swapper/300:31:493
364799205,11cyclictest0-21swapper/321:35:133
363399207,10cyclictest27201-21kthreadcore20:40:212
363399201,9cyclictest24958-21fschecks_count22:05:152
363399201,9cyclictest17010-21kthreadcore19:55:192
363399201,16cyclictest924-21snmpd20:00:222
363399201,10cyclictest8743-21fschecks_time19:15:162
363399201,10cyclictest23991-21cut21:45:232
361599207,10cyclictest23197-21df19:30:130
361599200,16cyclictest6624-21find20:20:110
364799199,8cyclictest0-21swapper/300:25:363
364799192,13cyclictest28366irq/58-eth021:45:133
364799192,11cyclictest0-21swapper/323:15:133
364799192,11cyclictest0-21swapper/322:47:413
364799192,11cyclictest0-21swapper/321:05:133
364799192,11cyclictest0-21swapper/320:55:143
364799192,11cyclictest0-21swapper/320:35:133
364799192,11cyclictest0-21swapper/320:15:143
364799192,10cyclictest0-21swapper/320:20:133
3647991912,2cyclictest0-21swapper/319:55:123
3647991911,5cyclictest28366irq/58-eth022:15:153
3647991911,4cyclictest0-21swapper/300:35:433
364799191,12cyclictest0-21swapper/319:40:173
3647991910,6cyclictest0-21swapper/323:55:023
363399193,8cyclictest12927-21/usr/sbin/munin21:00:112
364799189,6cyclictest0-21swapper/322:44:463
364799189,6cyclictest0-21swapper/300:12:423
364799189,6cyclictest0-21swapper/300:05:343
364799186,9cyclictest0-21swapper/321:50:133
364799184,11cyclictest0-21swapper/321:15:133
364799183,11cyclictest0-21swapper/322:29:413
364799183,10cyclictest0-21swapper/321:58:413
364799182,10cyclictest0-21swapper/323:00:143
364799182,10cyclictest0-21swapper/320:45:133
3647991812,4cyclictest0-21swapper/322:01:383
3647991812,4cyclictest0-21swapper/319:25:233
3647991811,4cyclictest0-21swapper/322:55:113
3647991810,6cyclictest0-21swapper/323:14:273
3647991810,6cyclictest0-21swapper/323:14:263
364799179,6cyclictest0-21swapper/323:55:143
364799172,10cyclictest0-21swapper/319:20:123
3647991711,4cyclictest321rcuc/319:30:113
3647991711,4cyclictest0-21swapper/321:00:143
3647991711,4cyclictest0-21swapper/320:10:263
3647991711,4cyclictest0-21swapper/320:00:143
3647991711,4cyclictest0-21swapper/300:01:563
364799171,12cyclictest0-21swapper/320:50:143
364799171,12cyclictest0-21swapper/300:20:143
364799161,9cyclictest0-21swapper/319:45:153
3647991611,3cyclictest0-21swapper/319:15:183
3647991610,4cyclictest0-21swapper/320:25:273
361599161,10cyclictest3727-21wc19:10:010
364799157,5cyclictest28366irq/58-eth022:50:133
364799151,12cyclictest0-21swapper/319:10:023
364799151,11cyclictest0-21swapper/320:10:023
3647991510,3cyclictest0-21swapper/320:30:173
3647991510,3cyclictest0-21swapper/319:35:243
364799149,3cyclictest0-21swapper/320:40:053
364799141,11cyclictest0-21swapper/319:10:143
363399130,10cyclictest3539-21latency_hist19:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional