You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 05:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Jul 18, 2026 00:45:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4197319910328,71cyclictest12-21ksoftirqd/000:00:060
419741997446,8cyclictest393508-21kworker/2:019:45:012
41973199742,40cyclictest131rcu_preempt00:34:010
41973199728,60cyclictest12-21ksoftirqd/000:24:450
41974199691,4cyclictest131rcu_preempt19:15:012
419731996912,34cyclictest16950irq/224-800000021:25:030
419731996728,4cyclictest12-21ksoftirqd/021:12:110
419731996728,4cyclictest12-21ksoftirqd/021:12:100
419731996718,44cyclictest12-21ksoftirqd/023:59:500
419731996618,43cyclictest12-21ksoftirqd/021:30:440
419731996618,43cyclictest12-21ksoftirqd/021:30:440
419731996517,43cyclictest12-21ksoftirqd/000:07:380
41973199645,43cyclictest12-21ksoftirqd/023:17:390
41973199637,28cyclictest12-21ksoftirqd/023:08:400
419731996229,3cyclictest12-21ksoftirqd/023:13:260
419731996215,4cyclictest12-21ksoftirqd/021:15:320
419731996210,6cyclictest131rcu_preempt00:15:010
419731996210,6cyclictest131rcu_preempt00:15:010
419731996127,3cyclictest4531351ssh22:07:000
419737996014,10cyclictest131rcu_preempt00:00:051
419741995934,7cyclictest425934-21kworker/2:322:23:012
419731995932,17cyclictest12-21ksoftirqd/022:30:020
419731995918,37cyclictest12-21ksoftirqd/022:40:030
419731995829,13cyclictest450579-21ssh22:01:290
419731995818,7cyclictest16850irq/134-800000000:26:090
419741995737,6cyclictest425934-21kworker/2:322:00:012
41973199572,39cyclictest435525-21ssh21:30:000
419731995722,31cyclictest12-21ksoftirqd/021:56:040
419731995719,6cyclictest12-21ksoftirqd/023:38:560
419731995719,34cyclictest16850irq/134-800000000:35:480
419731995718,35cyclictest16850irq/134-800000022:52:470
419731995718,34cyclictest16850irq/134-800000022:21:200
419731995712,41cyclictest12-21ksoftirqd/021:54:380
419741995641,7cyclictest393508-21kworker/2:019:25:022
419741995632,7cyclictest468765-21kworker/2:123:34:052
419741995632,7cyclictest468765-21kworker/2:123:34:042
419737995617,5cyclictest131rcu_preempt21:26:561
419731995618,7cyclictest16850irq/134-800000021:48:020
419731995618,34cyclictest16850irq/134-800000023:22:300
419731995618,34cyclictest16850irq/134-800000021:44:270
419731995618,34cyclictest16850irq/134-800000021:44:260
419731995617,35cyclictest16850irq/134-800000023:01:030
419731995617,34cyclictest16850irq/134-800000021:35:340
419745995537,13cyclictest3467602-21systemd-journal22:35:013
419741995539,12cyclictest275-21dbus-daemon21:30:012
419731995518,33cyclictest16850irq/134-800000000:17:030
419731995518,18cyclictest16850irq/134-800000022:30:020
419731995517,34cyclictest16850irq/134-800000023:32:090
419731995517,34cyclictest16850irq/134-800000023:32:090
419731995517,34cyclictest16850irq/134-800000022:59:270
419731995517,34cyclictest16850irq/134-800000022:11:520
419731995516,36cyclictest16850irq/134-800000022:43:020
419731995516,35cyclictest16850irq/134-800000023:50:350
419741995445,6cyclictest468765-21kworker/2:122:59:132
419731995418,6cyclictest16850irq/134-800000023:25:250
419731995417,33cyclictest16850irq/134-800000022:47:530
419731995417,33cyclictest16850irq/134-800000022:15:170
419731995417,33cyclictest16850irq/134-800000022:15:160
419731995416,34cyclictest16850irq/134-800000023:48:050
419731995416,34cyclictest16850irq/134-800000023:48:050
419741995346,4cyclictest454756-21kworker/2:000:37:552
419741995344,6cyclictest468765-21kworker/2:123:40:012
419741995344,6cyclictest454756-21kworker/2:000:17:152
419741995339,10cyclictest498397-21sshd23:42:222
419737995335,12cyclictest275-21dbus-daemon19:30:011
419737995325,20cyclictest275-21dbus-daemon20:45:011
419731995316,6cyclictest16850irq/134-800000023:40:140
419741995245,4cyclictest393508-21kworker/2:020:20:022
419741995244,5cyclictest454756-21kworker/2:023:20:482
419741995244,5cyclictest454756-21kworker/2:023:14:272
419741995244,5cyclictest425934-21kworker/2:322:18:332
419741995244,5cyclictest425934-21kworker/2:322:18:322
419741995244,5cyclictest425934-21kworker/2:321:54:132
419741995244,5cyclictest393508-21kworker/2:020:00:002
419741995243,6cyclictest468765-21kworker/2:123:04:372
419741995243,5cyclictest425934-21kworker/2:322:35:532
419741995242,6cyclictest393508-21kworker/2:021:32:502
419741995242,6cyclictest393508-21kworker/2:021:32:492
419731995222,21cyclictest12-21ksoftirqd/021:10:000
419741995143,5cyclictest468765-21kworker/2:123:58:442
419741995143,5cyclictest454756-21kworker/2:022:45:452
419741995143,5cyclictest393508-21kworker/2:021:40:122
419741995143,5cyclictest393508-21kworker/2:021:40:122
419741995143,5cyclictest393508-21kworker/2:021:10:022
419741995142,6cyclictest393508-21kworker/2:021:20:222
419741995142,6cyclictest393508-21kworker/2:021:18:472
419741995142,5cyclictest454756-21kworker/2:023:08:432
419737995134,12cyclictest3467602-21systemd-journal21:55:011
419737995119,21cyclictest23-21ksoftirqd/123:12:141
419731995139,7cyclictest419733-21cat19:10:020
419731995139,7cyclictest419733-21cat19:10:010
419731995123,20cyclictest12-21ksoftirqd/019:15:000
419731995116,3cyclictest12-21ksoftirqd/020:30:020
419731995116,3cyclictest12-21ksoftirqd/020:30:010
41974599504,3cyclictest131rcu_preempt00:18:213
419741995042,5cyclictest454756-21kworker/2:023:17:082
419741995042,5cyclictest425934-21kworker/2:322:33:592
419741995042,5cyclictest393508-21kworker/2:021:37:552
419741995042,5cyclictest393508-21kworker/2:021:14:202
419741995042,5cyclictest393508-21kworker/2:021:14:192
419741995041,6cyclictest468765-21kworker/2:123:26:502
419741995041,5cyclictest425934-21kworker/2:322:06:072
419741995040,6cyclictest425934-21kworker/2:321:49:162
419741995039,6cyclictest454756-21kworker/2:000:35:012
41973799507,5cyclictest131rcu_preempt22:05:521
419737995033,13cyclictest275-21dbus-daemon22:20:011
419737995033,13cyclictest275-21dbus-daemon22:20:001
419737995010,10cyclictest23-21ksoftirqd/100:34:321
419745994933,12cyclictest1-21systemd00:00:033
419745994931,13cyclictest275-21dbus-daemon19:45:013
419741994941,5cyclictest468765-21kworker/2:123:53:562
419741994941,5cyclictest468765-21kworker/2:123:47:032
419741994941,5cyclictest468765-21kworker/2:123:47:032
419741994941,5cyclictest468765-21kworker/2:122:53:512
419741994941,5cyclictest468765-21kworker/2:100:08:492
419741994941,5cyclictest468765-21kworker/2:100:03:262
419741994940,6cyclictest468765-21kworker/2:100:23:552
419741994940,5cyclictest454756-21kworker/2:022:40:192
41973799498,5cyclictest131rcu_preempt20:30:021
41973799498,5cyclictest131rcu_preempt20:30:021
419737994934,9cyclictest309-21in:imuxsock23:25:001
419737994929,16cyclictest3467602-21systemd-journal19:40:001
419737994927,16cyclictest275-21dbus-daemon21:25:011
419745994834,10cyclictest423027-21sshd20:13:233
419741994840,5cyclictest425934-21kworker/2:322:10:062
419741994839,6cyclictest468765-21kworker/2:100:10:112
419741994839,6cyclictest468765-21kworker/2:100:10:112
41974199482,3cyclictest131rcu_preempt19:20:012
419741994823,7cyclictest393508-21kworker/2:019:30:002
419741994817,21cyclictest30-21ksoftirqd/221:00:022
41973799489,8cyclictest23-21ksoftirqd/122:40:021
419737994826,16cyclictest275-21dbus-daemon20:40:011
419745994732,11cyclictest1-21systemd23:50:013
419745994732,11cyclictest1-21systemd23:50:003
419745994731,12cyclictest3467602-21systemd-journal20:20:023
419745994729,12cyclictest275-21dbus-daemon00:30:013
419745994720,17cyclictest275-21dbus-daemon21:45:013
419745994720,17cyclictest275-21dbus-daemon21:45:013
419745994718,6cyclictest425913-21diskmemload21:19:593
419741994739,5cyclictest425934-21kworker/2:322:29:492
419741994738,5cyclictest425934-21kworker/2:322:00:332
41973799478,25cyclictest423307-21modprobe20:20:011
41973799477,6cyclictest131rcu_preempt22:31:031
419737994731,11cyclictest275-21dbus-daemon21:40:011
41973799473,10cyclictest23-21ksoftirqd/100:05:021
419737994730,13cyclictest275-21dbus-daemon21:35:011
419737994730,13cyclictest275-21dbus-daemon21:35:001
419737994729,12cyclictest3467602-21systemd-journal22:25:001
419737994718,20cyclictest23-21ksoftirqd/123:02:541
41973799471,5cyclictest131rcu_preempt22:43:201
419737994710,5cyclictest131rcu_preempt00:05:101
419731994731,12cyclictest3467602-21systemd-journal20:55:010
419731994731,12cyclictest3467602-21systemd-journal20:55:000
41973199472,8cyclictest420196-21munin-run19:20:000
41973199471,3cyclictest131rcu_preempt19:35:010
41974199466,7cyclictest131rcu_preempt19:35:012
419741994638,5cyclictest393508-21kworker/2:020:50:012
41973799468,5cyclictest131rcu_preempt20:59:591
41973799468,10cyclictest23-21ksoftirqd/123:00:031
41973799466,7cyclictest23-21ksoftirqd/123:39:591
41973799466,30cyclictest476070-21cat22:55:021
419737994620,6cyclictest131rcu_preempt22:49:071
419737994619,19cyclictest23-21ksoftirqd/121:40:361
419737994619,19cyclictest23-21ksoftirqd/121:40:361
419737994618,20cyclictest23-21ksoftirqd/123:18:201
419737994610,6cyclictest131rcu_preempt21:57:531
41973799457,9cyclictest23-21ksoftirqd/121:20:001
419737994524,15cyclictest3467602-21systemd-journal23:50:011
419737994524,15cyclictest3467602-21systemd-journal23:50:011
419737994512,6cyclictest131rcu_preempt21:50:001
419737994510,6cyclictest131rcu_preempt19:50:021
419731994513,23cyclictest131rcu_preempt20:40:000
419745994432,9cyclictest516545-21sshd00:20:113
419745994428,13cyclictest275-21dbus-daemon23:55:013
41974599441,3cyclictest131rcu_preempt21:23:023
419741994436,5cyclictest393508-21kworker/2:020:35:022
419741994436,5cyclictest393508-21kworker/2:020:05:222
419741994436,5cyclictest393508-21kworker/2:019:52:242
41973799445,29cyclictest425905-21modprobe21:10:001
419737994425,14cyclictest3467602-21systemd-journal23:35:021
419737994425,14cyclictest3467602-21systemd-journal23:35:021
419737994417,5cyclictest131rcu_preempt19:25:011
419737994414,9cyclictest131rcu_preempt00:36:181
41973199445,33cyclictest423322-21modprobe20:20:010
41973199444,6cyclictest131rcu_preempt19:50:000
419731994437,4cyclictest16950irq/224-800000020:45:230
419731994437,4cyclictest16950irq/224-800000020:00:010
419731994428,12cyclictest12-21ksoftirqd/020:45:000
41973199442,6cyclictest131rcu_preempt20:15:010
41974599433,3cyclictest131rcu_preempt22:53:073
419745994327,12cyclictest37-21ksoftirqd/323:18:293
41974599432,11cyclictest37-21ksoftirqd/319:19:593
419741994335,5cyclictest393508-21kworker/2:021:05:012
419741994335,5cyclictest393508-21kworker/2:020:30:022
419741994335,5cyclictest393508-21kworker/2:020:30:012
419741994335,5cyclictest393508-21kworker/2:020:13:182
419741994335,5cyclictest393508-21kworker/2:019:35:132
41973799435,31cyclictest492348-21sh23:30:001
41973799434,4cyclictest131rcu_preempt22:12:201
41973799433,5cyclictest131rcu_preempt23:41:091
419737994329,11cyclictest23-21ksoftirqd/120:10:001
41973799432,4cyclictest131rcu_preempt00:15:281
419731994320,5cyclictest12-21ksoftirqd/021:00:010
419731994319,4cyclictest12-21ksoftirqd/021:05:000
41974599423,15cyclictest3467602-21systemd-journal20:35:003
419745994225,13cyclictest275-21dbus-daemon19:25:483
41974599422,3cyclictest131rcu_preempt21:54:263
419741994235,5cyclictest454756-21kworker/2:000:26:352
41974199421,3cyclictest131rcu_preempt20:40:012
41973799426,5cyclictest131rcu_preempt00:24:571
419737994230,9cyclictest23-21ksoftirqd/122:00:581
41973799422,6cyclictest131rcu_preempt21:05:021
419737994224,13cyclictest0-21swapper/120:35:011
419737994215,18cyclictest275-21dbus-daemon20:00:011
419737994213,21cyclictest275-21dbus-daemon20:55:011
419737994213,21cyclictest275-21dbus-daemon20:55:011
41973199423,6cyclictest131rcu_preempt19:25:010
419731994219,5cyclictest12-21ksoftirqd/019:40:020
419745994130,8cyclictest37-21ksoftirqd/323:13:213
419745994129,8cyclictest425900-21cron21:09:593
419745994127,11cyclictest275-21dbus-daemon20:10:013
41974599412,3cyclictest131rcu_preempt22:06:173
41974599412,3cyclictest131rcu_preempt00:39:543
419741994133,5cyclictest393508-21kworker/2:020:20:292
41973799415,5cyclictest131rcu_preempt00:25:151
41973799414,26cyclictest419738-21latency_hist19:10:021
41973799414,26cyclictest419738-21latency_hist19:10:011
419737994131,7cyclictest23-21ksoftirqd/119:20:011
419737994128,11cyclictest23-21ksoftirqd/122:30:011
419737994125,11cyclictest23-21ksoftirqd/123:05:471
41974599405,8cyclictest37-21ksoftirqd/322:21:503
41974599403,3cyclictest131rcu_preempt23:40:003
41974599401,8cyclictest37-21ksoftirqd/322:02:493
41974599401,3cyclictest131rcu_preempt21:28:133
41974599401,3cyclictest131rcu_preempt00:10:213
41974599401,3cyclictest131rcu_preempt00:10:213
419741994031,6cyclictest393508-21kworker/2:020:45:012
41973799408,5cyclictest131rcu_preempt00:14:491
41973799408,5cyclictest131rcu_preempt00:14:491
419737994012,5cyclictest131rcu_preempt21:10:511
419737994012,5cyclictest131rcu_preempt21:10:511
419737994010,5cyclictest131rcu_preempt19:55:021
41974599398,5cyclictest131rcu_preempt22:32:573
41974599396,3cyclictest131rcu_preempt19:10:023
41974599396,3cyclictest131rcu_preempt19:10:013
419745993926,10cyclictest471191-21sed22:45:023
419745993926,10cyclictest443343-21ssh21:46:193
41974599392,3cyclictest131rcu_preempt21:30:543
41974599392,3cyclictest131rcu_preempt21:30:533
41974599391,4cyclictest131rcu_preempt23:02:013
41974599391,3cyclictest131rcu_preempt22:55:073
41974599391,3cyclictest131rcu_preempt00:09:153
41974199394,3cyclictest131rcu_preempt19:50:022
41973799399,5cyclictest131rcu_preempt20:25:011
41973799399,4cyclictest131rcu_preempt20:12:221
41973799395,24cyclictest422564-21grep20:05:021
41973799395,24cyclictest422564-21grep20:05:021
41973799394,5cyclictest131rcu_preempt23:52:101
419731993932,4cyclictest16950irq/224-800000020:05:010
419731993932,4cyclictest16950irq/224-800000020:05:010
41973199391,33cyclictest0-21swapper/019:43:430
419745993824,11cyclictest37-21ksoftirqd/323:10:013
41974599382,3cyclictest131rcu_preempt00:00:253
419745993823,12cyclictest37-21ksoftirqd/320:45:023
419745993817,15cyclictest275-21dbus-daemon20:05:013
419745993817,15cyclictest275-21dbus-daemon20:05:003
41974599381,6cyclictest455993-21ssh22:13:113
419745993815,5cyclictest131rcu_preempt21:39:593
419745993814,7cyclictest131rcu_preempt22:20:013
419745993814,7cyclictest131rcu_preempt22:20:013
41974599381,3cyclictest131rcu_preempt23:24:353
41974599381,3cyclictest131rcu_preempt22:29:133
41974199384,3cyclictest131rcu_preempt20:05:012
41974199384,3cyclictest131rcu_preempt20:05:012
41974199381,3cyclictest131rcu_preempt19:10:022
41974199381,3cyclictest131rcu_preempt19:10:012
41973799389,5cyclictest131rcu_preempt19:15:021
41973799384,4cyclictest131rcu_preempt19:35:011
41973199389,25cyclictest421766-21kworker/0:219:52:440
419731993829,5cyclictest12-21ksoftirqd/019:30:010
41974599375,8cyclictest37-21ksoftirqd/321:14:223
41974599375,8cyclictest37-21ksoftirqd/321:14:223
419745993728,7cyclictest37-21ksoftirqd/323:41:183
419745993724,10cyclictest3467602-21systemd-journal19:15:013
419745993723,11cyclictest37-21ksoftirqd/323:28:103
419745993723,10cyclictest421992-21sshd19:54:223
419745993722,10cyclictest3467602-21systemd-journal21:55:233
419745993720,13cyclictest310-21in:imklog22:47:143
41974199374,3cyclictest131rcu_preempt20:55:012
41974199374,3cyclictest131rcu_preempt20:55:002
41973799374,22cyclictest421505-21munin-run19:45:001
419737993727,8cyclictest23-21ksoftirqd/120:46:051
419731993726,8cyclictest3467602-21systemd-journal20:07:210
419745993623,9cyclictest1-21systemd19:50:003
41974599361,3cyclictest131rcu_preempt20:55:013
41974599361,3cyclictest131rcu_preempt20:55:003
41973199362,6cyclictest405835-21munin-node20:20:210
419745993526,6cyclictest493429-21sh23:32:023
419745993526,6cyclictest493429-21sh23:32:023
419745993521,8cyclictest309-21in:imuxsock19:35:003
419745993519,12cyclictest0-21swapper/320:19:273
41974599351,4cyclictest131rcu_preempt00:30:533
419745993425,7cyclictest37-21ksoftirqd/320:50:013
419745993423,9cyclictest37-21ksoftirqd/319:40:013
419745993423,8cyclictest37-21ksoftirqd/321:04:193
419731993428,4cyclictest0-21swapper/020:31:130
41974599334,4cyclictest131rcu_preempt20:00:003
419745993324,6cyclictest425405-21sendmail-msp21:00:003
419745993224,5cyclictest423813-21sshd20:26:383
419745993224,5cyclictest423813-21sshd20:26:383
419745993222,6cyclictest424510-21latency_hist20:40:023
41974599311,3cyclictest131rcu_preempt19:25:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional