You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 11:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Apr 04, 2026 00:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4790859911535,62cyclictest16850irq/134-800000022:35:020
479088998810,57cyclictest574055-21ssh00:35:021
47908599871,13cyclictest565492-21latency_hist00:15:030
479085998625,15cyclictest498814-21ssh21:40:560
479085998625,15cyclictest498814-21ssh21:40:550
479085998524,12cyclictest558775-21ssh23:59:420
479085998428,11cyclictest573364-21ssh00:33:310
479085998327,12cyclictest487329-21ssh21:14:540
479085998320,58cyclictest16850irq/134-800000023:37:060
479095998126,40cyclictest552427-21kworker/u8:123:50:022
479085998118,58cyclictest16850irq/134-800000023:47:240
479085998029,17cyclictest506773-21ssh21:59:440
479085998019,51cyclictest16850irq/134-800000023:11:020
479100997926,14cyclictest3467602-21systemd-journal00:25:033
479085997834,38cyclictest1-21systemd22:15:010
479085997827,15cyclictest518118-21sshd22:25:150
479085997816,56cyclictest16850irq/134-800000000:03:080
479085997712,55cyclictest16850irq/134-800000023:31:590
479085997712,55cyclictest16850irq/134-800000023:31:580
479085997528,15cyclictest536754-21ssh23:08:450
479085997524,13cyclictest493190-21ssh21:28:070
479085997524,11cyclictest542232-21ssh23:21:070
479085997517,52cyclictest16850irq/134-800000000:21:250
479085997413,23cyclictest489603-21munin-run21:20:010
479085997413,23cyclictest489603-21munin-run21:20:000
479095997356,12cyclictest0-21swapper/219:10:022
479085997327,12cyclictest527026-21ssh22:45:580
479085997325,12cyclictest501042-21ssh21:45:590
47908599731,30cyclictest3467602-21systemd-journal23:01:050
47908599731,30cyclictest3467602-21systemd-journal23:01:040
479085997311,56cyclictest16850irq/134-800000000:08:270
479095997210,6cyclictest131rcu_preempt23:10:012
479085997227,14cyclictest515489-21ssh22:19:370
479085997223,12cyclictest551578-21ssh23:42:480
47908899717,11cyclictest23-21ksoftirqd/100:00:021
479088997116,15cyclictest23-21ksoftirqd/100:20:001
479085997129,3cyclictest0-21swapper/023:00:010
479085997127,13cyclictest529297-21ssh22:51:090
479085997116,50cyclictest16850irq/134-800000022:23:380
479085997029,4cyclictest16850irq/134-800000022:00:030
479085997021,14cyclictest574529-21ssh00:35:540
479085996916,48cyclictest16850irq/134-800000000:16:200
479085996828,11cyclictest12-21ksoftirqd/000:28:190
479085996813,50cyclictest16850irq/134-800000021:38:500
47909599676,6cyclictest131rcu_preempt00:00:012
47909599665,5cyclictest131rcu_preempt21:11:542
47909599662,36cyclictest480571-21cron19:39:592
479085996618,43cyclictest16850irq/134-800000023:19:230
479085996510,6cyclictest16850irq/134-800000023:54:290
479085996430,4cyclictest16950irq/224-800000020:15:560
479085996414,45cyclictest16850irq/134-800000021:34:410
479085996410,48cyclictest16850irq/134-800000021:21:020
479095996353,6cyclictest484201-21kworker/2:122:53:422
479095996319,14cyclictest275-21dbus-daemon21:00:012
479100996228,8cyclictest37-21ksoftirqd/322:45:023
47908599615,31cyclictest16850irq/134-800000023:25:170
479085996123,11cyclictest484750-21systemctl21:00:000
47908599611,31cyclictest524722-21sh22:40:330
479095996026,6cyclictest489909-21sshd21:20:152
479088996025,13cyclictest23-21ksoftirqd/123:05:011
479088996025,13cyclictest23-21ksoftirqd/123:05:001
47908599609,6cyclictest16850irq/134-800000022:07:200
47908599609,6cyclictest16850irq/134-800000022:07:190
479085996024,11cyclictest275-21dbus-daemon21:55:020
479085996024,11cyclictest275-21dbus-daemon21:55:020
479088995839,15cyclictest1-21systemd00:25:011
47908899581,8cyclictest0-21swapper/119:20:021
479085995829,6cyclictest16950irq/224-800000022:35:220
479100995738,15cyclictest3467602-21systemd-journal22:04:593
479088995734,19cyclictest3467602-21systemd-journal19:39:591
479088995617,15cyclictest23-21ksoftirqd/122:00:001
479088995615,8cyclictest23-21ksoftirqd/123:19:591
47908899547,9cyclictest23-21ksoftirqd/123:30:011
479088995411,5cyclictest131rcu_preempt23:47:151
479088995326,18cyclictest275-21dbus-daemon21:15:001
479088995317,10cyclictest23-21ksoftirqd/100:40:021
47909599527,10cyclictest30-21ksoftirqd/221:52:472
47909599527,10cyclictest30-21ksoftirqd/221:52:472
479095995235,12cyclictest1-21systemd23:00:012
479095995228,4cyclictest538305-21sh23:12:132
47909599522,25cyclictest0-21swapper/222:16:352
47908899526,9cyclictest23-21ksoftirqd/100:15:001
479088995244,6cyclictest23-21ksoftirqd/123:34:121
479088995244,6cyclictest23-21ksoftirqd/123:34:111
479088995217,8cyclictest23-21ksoftirqd/123:22:101
479088995212,7cyclictest131rcu_preempt23:00:021
47909599514,4cyclictest131rcu_preempt23:35:012
47909599514,4cyclictest131rcu_preempt23:35:012
479095995143,5cyclictest552519-21kworker/2:200:19:492
479095995143,5cyclictest552519-21kworker/2:200:03:552
479095995143,5cyclictest484201-21kworker/2:122:49:372
479095995143,5cyclictest484201-21kworker/2:122:21:492
47908899516,11cyclictest23-21ksoftirqd/121:30:001
47908899514,11cyclictest23-21ksoftirqd/121:52:221
47908899514,11cyclictest23-21ksoftirqd/121:52:211
479088995110,6cyclictest131rcu_preempt00:26:251
479095995042,5cyclictest552519-21kworker/2:200:36:022
479095995042,5cyclictest500449-21kworker/2:322:33:272
479095995042,5cyclictest484201-21kworker/2:122:11:082
479095995041,5cyclictest484201-21kworker/2:123:42:042
479095995034,13cyclictest1-21systemd00:30:012
47908899506,8cyclictest23-21ksoftirqd/122:42:451
47908899503,8cyclictest23-21ksoftirqd/120:00:021
479088995031,14cyclictest3467602-21systemd-journal20:50:001
479088995018,13cyclictest23-21ksoftirqd/122:09:401
479088995018,13cyclictest23-21ksoftirqd/122:09:401
479088995014,9cyclictest131rcu_preempt22:22:161
47908599502,3cyclictest131rcu_preempt19:40:020
479100994933,13cyclictest498244-21run-parts21:40:003
479100994914,6cyclictest131rcu_preempt23:00:033
479095994941,5cyclictest500449-21kworker/2:322:29:132
479095994941,5cyclictest484201-21kworker/2:122:44:462
479095994941,5cyclictest483949-21kworker/2:021:32:302
47908899496,12cyclictest23-21ksoftirqd/120:10:011
479088994939,7cyclictest23-21ksoftirqd/122:25:361
479088994939,7cyclictest23-21ksoftirqd/100:05:371
479088994932,12cyclictest482671-21systemctl20:20:001
479088994916,22cyclictest221rcuc/123:12:591
479088994915,6cyclictest131rcu_preempt22:55:021
47910099483,7cyclictest131rcu_preempt22:00:003
47910099482,8cyclictest37-21ksoftirqd/322:20:423
479095994840,5cyclictest552519-21kworker/2:200:30:212
479095994840,5cyclictest484201-21kworker/2:123:03:392
479095994840,5cyclictest484201-21kworker/2:123:03:382
479095994840,5cyclictest483949-21kworker/2:021:26:022
479095994840,5cyclictest479301-21kworker/2:119:46:052
479088994833,11cyclictest23-21ksoftirqd/119:55:001
479088994829,13cyclictest23-21ksoftirqd/121:00:001
479088994826,17cyclictest3467602-21systemd-journal22:20:011
47908899481,8cyclictest23-21ksoftirqd/122:03:541
479088994816,6cyclictest131rcu_preempt21:41:241
479088994816,6cyclictest131rcu_preempt21:41:231
479088994815,6cyclictest131rcu_preempt21:20:011
479088994815,6cyclictest131rcu_preempt21:20:001
47908599482,42cyclictest479090-21latency_hist19:10:020
47910099475,8cyclictest37-21ksoftirqd/321:10:033
47910099475,8cyclictest37-21ksoftirqd/321:10:023
47910099475,3cyclictest150-21pr/ttyS220:55:023
479095994740,4cyclictest484201-21kworker/2:123:38:342
479095994739,5cyclictest552519-21kworker/2:200:21:212
479095994733,11cyclictest1-21systemd00:10:012
47908899479,9cyclictest131rcu_preempt22:30:311
47908899476,7cyclictest131rcu_preempt23:10:021
47908899476,10cyclictest23-21ksoftirqd/123:39:531
479088994720,10cyclictest23-21ksoftirqd/123:44:151
479088994714,8cyclictest23-21ksoftirqd/123:53:211
479088994713,15cyclictest131rcu_preempt21:36:021
479088994710,5cyclictest131rcu_preempt22:49:491
47910099468,4cyclictest131rcu_preempt23:10:023
47910099463,6cyclictest37-21ksoftirqd/322:12:263
479095994639,4cyclictest484201-21kworker/2:121:47:142
479095994638,5cyclictest500449-21kworker/2:321:59:592
479095994638,4cyclictest484201-21kworker/2:122:39:402
479095994637,5cyclictest484201-21kworker/2:123:29:162
479095994637,5cyclictest483949-21kworker/2:021:37:042
47909599461,5cyclictest131rcu_preempt22:00:032
47908899467,5cyclictest131rcu_preempt22:12:181
47908899466,5cyclictest131rcu_preempt20:40:011
47908899466,5cyclictest131rcu_preempt20:40:011
479088994616,5cyclictest131rcu_preempt21:49:341
479088994613,6cyclictest131rcu_preempt22:36:561
479085994630,12cyclictest3467602-21systemd-journal19:56:060
479085994628,14cyclictest12-21ksoftirqd/020:40:020
479085994628,14cyclictest12-21ksoftirqd/020:40:020
479100994528,13cyclictest275-21dbus-daemon19:40:013
47910099452,4cyclictest131rcu_preempt22:34:153
479095994538,4cyclictest484201-21kworker/2:121:05:022
479095994537,5cyclictest479301-21kworker/2:120:05:022
479095994536,6cyclictest552519-21kworker/2:200:12:032
479095994530,12cyclictest275-21dbus-daemon23:25:002
47909599452,3cyclictest131rcu_preempt23:20:022
47908899455,7cyclictest23-21ksoftirqd/121:34:001
479088994534,7cyclictest23-21ksoftirqd/100:04:011
47908899453,10cyclictest23-21ksoftirqd/121:22:301
479088994512,6cyclictest131rcu_preempt20:29:591
47910099445,4cyclictest131rcu_preempt23:20:003
47910099444,3cyclictest150-21pr/ttyS200:07:403
47910099444,3cyclictest131rcu_preempt23:30:003
47910099443,4cyclictest131rcu_preempt21:45:033
47910099443,4cyclictest131rcu_preempt21:45:023
47910099441,5cyclictest131rcu_preempt19:35:003
47909599446,4cyclictest131rcu_preempt20:20:022
47909599444,3cyclictest131rcu_preempt20:35:002
479095994436,5cyclictest484201-21kworker/2:121:45:022
479095994436,5cyclictest484201-21kworker/2:121:45:022
479088994429,10cyclictest309-21in:imuxsock19:11:121
479088994425,14cyclictest3467602-21systemd-journal20:04:591
479085994437,4cyclictest16850irq/134-800000021:10:010
479085994437,4cyclictest16850irq/134-800000021:10:000
47908599442,3cyclictest131rcu_preempt19:50:010
47908599441,38cyclictest3467602-21systemd-journal20:06:090
47910099437,6cyclictest131rcu_preempt22:20:013
47910099436,26cyclictest530775-21ssh22:55:013
47910099435,5cyclictest131rcu_preempt00:37:043
47910099433,35cyclictest309-21in:imuxsock21:05:013
47910099432,8cyclictest131rcu_preempt23:15:023
47910099432,6cyclictest131rcu_preempt21:19:063
47910099432,6cyclictest131rcu_preempt21:19:063
47910099431,6cyclictest131rcu_preempt20:00:003
47910099431,4cyclictest131rcu_preempt21:13:593
47910099431,3cyclictest131rcu_preempt00:05:003
479100994310,5cyclictest131rcu_preempt19:20:013
479095994337,4cyclictest500449-21kworker/2:322:05:452
479095994337,4cyclictest500449-21kworker/2:322:05:442
47909599433,3cyclictest131rcu_preempt23:53:492
47909599433,3cyclictest131rcu_preempt20:50:022
47909599432,3cyclictest131rcu_preempt19:30:012
479088994333,6cyclictest364-21snmpd20:32:051
47908899431,8cyclictest23-21ksoftirqd/121:10:021
47908899431,8cyclictest23-21ksoftirqd/121:10:011
47908599438,4cyclictest131rcu_preempt20:50:020
47910099426,6cyclictest37-21ksoftirqd/323:01:403
47910099426,6cyclictest37-21ksoftirqd/323:01:393
479100994227,10cyclictest275-21dbus-daemon19:55:013
479100994227,10cyclictest275-21dbus-daemon00:00:013
479100994226,11cyclictest3467602-21systemd-journal22:45:033
47910099421,4cyclictest131rcu_preempt21:50:273
47910099421,4cyclictest131rcu_preempt21:50:263
47910099421,4cyclictest131rcu_preempt00:17:013
479095994233,5cyclictest483707-21sendmail20:40:012
479095994233,5cyclictest483707-21sendmail20:40:002
479095994228,10cyclictest1-21systemd19:20:022
479088994229,9cyclictest3467602-21systemd-journal19:24:591
479088994224,13cyclictest275-21dbus-daemon19:45:021
47908899421,3cyclictest131rcu_preempt20:45:011
479085994234,5cyclictest16950irq/224-800000019:20:000
479085994226,12cyclictest481335-21sshd19:51:110
47910099419,6cyclictest131rcu_preempt19:30:013
47910099416,7cyclictest37-21ksoftirqd/322:35:403
47910099414,4cyclictest131rcu_preempt22:26:173
47910099413,4cyclictest131rcu_preempt21:28:093
479100994131,6cyclictest550209-21cron23:40:003
479100994129,9cyclictest37-21ksoftirqd/320:40:013
479100994129,9cyclictest37-21ksoftirqd/320:40:003
479100994128,9cyclictest1-21systemd22:10:013
479100994128,9cyclictest1-21systemd22:10:013
479100994127,10cyclictest309-21in:imuxsock00:24:513
47910099411,5cyclictest131rcu_preempt19:45:003
47910099411,4cyclictest131rcu_preempt23:25:023
47910099411,3cyclictest131rcu_preempt19:10:023
479095994131,7cyclictest483159-21sshd20:25:152
47909599412,3cyclictest131rcu_preempt21:10:022
47909599412,3cyclictest131rcu_preempt21:10:012
479088994130,9cyclictest23-21ksoftirqd/120:25:031
479088994130,9cyclictest23-21ksoftirqd/120:25:021
47908599415,4cyclictest131rcu_preempt19:35:010
47908599412,4cyclictest131rcu_preempt20:00:120
47910099407,6cyclictest131rcu_preempt23:43:533
47910099405,4cyclictest131rcu_preempt19:15:013
47910099403,4cyclictest131rcu_preempt21:24:313
47910099403,4cyclictest131rcu_preempt00:33:493
47910099402,5cyclictest131rcu_preempt21:45:213
479100994010,5cyclictest131rcu_preempt23:52:043
479095994032,5cyclictest484201-21kworker/2:121:15:392
479095994032,5cyclictest484201-21kworker/2:121:15:382
479095994030,7cyclictest479792-21sshd19:20:102
47909599402,3cyclictest131rcu_preempt20:55:002
47909599402,3cyclictest131rcu_preempt19:53:102
47909599401,4cyclictest131rcu_preempt19:35:012
47909599401,3cyclictest131rcu_preempt20:10:002
47909599401,3cyclictest131rcu_preempt19:58:122
479088994032,6cyclictest23-21ksoftirqd/121:00:021
479088994016,5cyclictest131rcu_preempt19:50:001
47908599403,3cyclictest131rcu_preempt20:30:020
479085994026,10cyclictest480066-21cron19:30:000
47908599401,6cyclictest0-21swapper/019:45:010
47910099395,5cyclictest131rcu_preempt00:11:543
47910099394,8cyclictest37-21ksoftirqd/320:14:583
47910099393,4cyclictest131rcu_preempt23:35:023
47910099393,4cyclictest131rcu_preempt23:35:023
479100993925,9cyclictest309-21in:imuxsock20:34:163
47910099392,4cyclictest131rcu_preempt23:50:023
479085993930,6cyclictest484486-21munin-plugin-st20:55:000
479085993930,6cyclictest19350irq/34-mmc120:41:550
479085993927,8cyclictest3467602-21systemd-journal20:23:160
479085993927,8cyclictest3467602-21systemd-journal20:23:150
479085993915,20cyclictest16850irq/134-800000019:20:110
47910099383,6cyclictest131rcu_preempt20:45:013
47910099382,5cyclictest131rcu_preempt21:34:393
47910099381,5cyclictest131rcu_preempt20:30:003
47909599382,4cyclictest131rcu_preempt20:21:122
47909599382,4cyclictest131rcu_preempt20:21:112
47908899389,5cyclictest131rcu_preempt19:30:011
479088993829,5cyclictest309-21in:imuxsock19:35:001
479085993828,7cyclictest12-21ksoftirqd/020:15:030
479085993826,8cyclictest479486-21latency_hist19:15:020
479100993727,7cyclictest482117-21sshd20:06:153
479100993724,9cyclictest3467602-21systemd-journal20:05:003
479095993729,5cyclictest479301-21kworker/2:119:45:012
47908599374,4cyclictest131rcu_preempt20:35:010
47910099364,3cyclictest131rcu_preempt20:18:153
479100993617,3cyclictest37-21ksoftirqd/319:21:113
479088993626,7cyclictest23-21ksoftirqd/120:15:001
479088993622,11cyclictest3467602-21systemd-journal20:53:021
479085993621,10cyclictest309-21in:imuxsock21:00:020
47910099354,3cyclictest131rcu_preempt21:00:003
47910099354,3cyclictest131rcu_preempt20:50:023
47910099352,4cyclictest131rcu_preempt19:50:003
47909599353,3cyclictest131rcu_preempt20:45:012
479095993529,4cyclictest482397-21sh20:15:012
47908899359,5cyclictest131rcu_preempt19:10:011
47910099343,3cyclictest131rcu_preempt20:25:033
47910099343,3cyclictest131rcu_preempt20:25:023
479095993417,12cyclictest3467602-21systemd-journal19:15:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional