You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 15:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Thu Jul 30, 2026 00:45:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
151664997933,11cyclictest275-21dbus-daemon23:45:003
151664996932,8cyclictest157337-21sshd20:59:353
151661996941,10cyclictest157112-21kworker/2:023:08:552
151659996825,34cyclictest235926-21ssh23:53:230
151659996824,23cyclictest165480-21ssh21:25:280
151659996820,43cyclictest16850irq/134-800000022:19:590
151659996424,36cyclictest12-21ksoftirqd/023:22:150
15166099637,11cyclictest23-21ksoftirqd/121:55:031
151659996323,18cyclictest12-21ksoftirqd/000:14:350
151661996236,9cyclictest157112-21kworker/2:023:01:182
151661996210,11cyclictest30-21ksoftirqd/222:12:002
151660996221,13cyclictest23-21ksoftirqd/120:20:001
151660996216,8cyclictest23-21ksoftirqd/100:24:391
151660996124,9cyclictest23-21ksoftirqd/122:41:431
151659996134,11cyclictest205759-21ssh22:50:010
151659996123,29cyclictest183590-21ssh22:03:340
151659996123,29cyclictest183590-21ssh22:03:340
151659996121,35cyclictest16850irq/134-800000021:10:010
151659996117,38cyclictest16850irq/134-800000000:05:020
151660996016,10cyclictest23-21ksoftirqd/121:30:011
151659996028,23cyclictest248212-21rm00:19:080
151659996023,15cyclictest178256-21rm21:52:200
151659996016,34cyclictest16850irq/134-800000023:30:000
15166099598,7cyclictest23-21ksoftirqd/121:31:121
15166099598,7cyclictest23-21ksoftirqd/121:31:121
15166099595,9cyclictest23-21ksoftirqd/121:14:391
151660995913,10cyclictest23-21ksoftirqd/100:10:021
151659995941,13cyclictest1-21systemd20:55:000
151659995929,14cyclictest200102-21ssh22:38:140
151659995929,14cyclictest200102-21ssh22:38:140
151659995925,22cyclictest180165-21ssh21:56:140
151661995834,8cyclictest157112-21kworker/2:023:56:032
151659995819,18cyclictest197232-21ssh22:32:000
15166099577,10cyclictest207183-21ssh22:52:561
15166099576,9cyclictest23-21ksoftirqd/122:07:251
15165999572,4cyclictest131rcu_preempt21:39:240
151661995646,6cyclictest157112-21kworker/2:000:13:302
151661995636,7cyclictest126577-21kworker/2:221:32:092
151661995636,7cyclictest126577-21kworker/2:221:32:092
15166099569,14cyclictest23-21ksoftirqd/122:15:021
15166099567,14cyclictest23-21ksoftirqd/121:19:381
15166099564,11cyclictest23-21ksoftirqd/121:50:011
151660995617,10cyclictest23-21ksoftirqd/123:05:011
151660995612,11cyclictest23-21ksoftirqd/100:39:011
151660995612,11cyclictest23-21ksoftirqd/100:39:011
151660995611,7cyclictest23-21ksoftirqd/119:20:021
151659995627,20cyclictest12-21ksoftirqd/000:25:090
151659995627,20cyclictest12-21ksoftirqd/000:25:090
151659995617,35cyclictest16850irq/134-800000023:57:140
151664995534,17cyclictest275-21dbus-daemon00:40:003
151664995534,17cyclictest275-21dbus-daemon00:40:003
151661995537,12cyclictest1-21systemd00:20:002
151661995529,8cyclictest157112-21kworker/2:022:59:382
15166099558,15cyclictest23-21ksoftirqd/100:25:111
15166099558,15cyclictest23-21ksoftirqd/100:25:111
15166099556,12cyclictest23-21ksoftirqd/121:24:021
15166099556,12cyclictest23-21ksoftirqd/121:24:011
15166099556,11cyclictest23-21ksoftirqd/122:04:331
15166099556,11cyclictest23-21ksoftirqd/122:04:331
151660995518,8cyclictest23-21ksoftirqd/100:15:071
151660995513,9cyclictest23-21ksoftirqd/122:47:471
151660995511,14cyclictest23-21ksoftirqd/121:40:011
151659995529,22cyclictest163236-21ssh21:20:470
151659995529,22cyclictest163236-21ssh21:20:470
151659995528,11cyclictest254563-21ssh00:32:150
151659995528,11cyclictest228069-21ssh23:36:410
151659995528,11cyclictest228069-21ssh23:36:410
151659995528,11cyclictest185183-21ssh22:06:430
15165999552,4cyclictest131rcu_preempt22:29:140
151659995522,24cyclictest188811-21ssh22:14:370
151659995521,30cyclictest12-21ksoftirqd/023:44:570
15165999551,42cyclictest12-21ksoftirqd/021:48:560
151661995446,5cyclictest157112-21kworker/2:021:40:002
151661995444,6cyclictest157112-21kworker/2:021:48:112
15166099549,9cyclictest23-21ksoftirqd/123:35:011
15166099549,9cyclictest23-21ksoftirqd/123:25:091
15166099549,7cyclictest23-21ksoftirqd/122:32:081
15166099546,14cyclictest23-21ksoftirqd/123:50:001
15166099545,9cyclictest23-21ksoftirqd/123:40:011
15166099545,9cyclictest23-21ksoftirqd/123:40:011
15166099545,15cyclictest174105-21sh21:43:531
15166099545,15cyclictest174105-21sh21:43:531
151660995431,16cyclictest3467602-21systemd-journal20:05:001
151659995444,7cyclictest12-21ksoftirqd/021:10:100
151659995427,12cyclictest225689-21ssh23:31:380
151659995425,13cyclictest213711-21ssh23:06:280
151659995424,15cyclictest169887-21ssh21:35:010
151659995424,15cyclictest169887-21ssh21:35:010
151659995413,36cyclictest16850irq/134-800000023:14:010
151664995328,17cyclictest275-21dbus-daemon22:45:003
151661995345,5cyclictest157112-21kworker/2:023:27:402
151661995344,6cyclictest157112-21kworker/2:021:14:352
151661995344,5cyclictest157112-21kworker/2:023:20:012
151661995344,5cyclictest157112-21kworker/2:022:31:012
151661995343,6cyclictest201708-21kworker/2:200:08:312
151661995342,6cyclictest157112-21kworker/2:023:14:042
151661995329,8cyclictest157112-21kworker/2:022:00:032
151661995329,8cyclictest157112-21kworker/2:022:00:032
15166099539,11cyclictest23-21ksoftirqd/123:20:071
15166099536,14cyclictest240899-21ssh00:03:491
15166099535,9cyclictest23-21ksoftirqd/123:08:571
15166099535,11cyclictest23-21ksoftirqd/100:11:231
151660995330,17cyclictest3467602-21systemd-journal23:00:001
151660995317,8cyclictest23-21ksoftirqd/122:21:341
151660995313,11cyclictest23-21ksoftirqd/123:20:471
151660995311,8cyclictest23-21ksoftirqd/122:38:031
151660995311,8cyclictest23-21ksoftirqd/122:38:031
15166099531,14cyclictest23-21ksoftirqd/123:43:291
15166099531,14cyclictest23-21ksoftirqd/121:09:591
151660995310,7cyclictest23-21ksoftirqd/123:12:541
15165999534,3cyclictest131rcu_preempt22:40:080
15165999531,47cyclictest3467602-21systemd-journal21:45:010
15165999531,47cyclictest3467602-21systemd-journal21:45:000
15166499526,34cyclictest361rcuc/320:10:023
151661995244,5cyclictest201708-21kworker/2:200:04:162
151661995244,5cyclictest157112-21kworker/2:023:46:202
151661995244,5cyclictest157112-21kworker/2:022:16:042
151661995244,5cyclictest157112-21kworker/2:000:22:242
151661995243,5cyclictest246784-21kworker/2:100:38:442
151661995243,5cyclictest246784-21kworker/2:100:38:442
151661995243,5cyclictest157112-21kworker/2:022:25:142
151661995243,5cyclictest157112-21kworker/2:021:53:362
151661995243,5cyclictest157112-21kworker/2:021:41:302
151661995243,5cyclictest157112-21kworker/2:021:41:302
151661995243,5cyclictest157112-21kworker/2:021:20:592
151661995243,5cyclictest157112-21kworker/2:021:20:582
151661995234,13cyclictest275-21dbus-daemon21:25:482
151661995212,4cyclictest131rcu_preempt23:36:552
151661995212,4cyclictest131rcu_preempt23:36:552
15166099526,9cyclictest23-21ksoftirqd/122:27:021
15166099524,6cyclictest23-21ksoftirqd/123:55:171
15166099524,11cyclictest23-21ksoftirqd/120:40:001
151660995236,12cyclictest3467602-21systemd-journal21:00:001
151660995214,9cyclictest23-21ksoftirqd/122:00:001
151659995228,20cyclictest208007-21ssh22:54:480
151659995212,31cyclictest16850irq/134-800000000:05:030
151661995143,5cyclictest157112-21kworker/2:023:21:192
151661995143,5cyclictest157112-21kworker/2:022:23:442
151661995143,5cyclictest157112-21kworker/2:021:58:132
151661995141,6cyclictest157112-21kworker/2:023:52:022
151661995141,6cyclictest157112-21kworker/2:022:53:272
15166099517,9cyclictest23-21ksoftirqd/100:32:441
15166099515,9cyclictest23-21ksoftirqd/119:40:001
151661995042,5cyclictest201708-21kworker/2:223:41:172
151661995042,5cyclictest157112-21kworker/2:000:34:562
151661995041,6cyclictest157112-21kworker/2:022:41:072
151661995041,5cyclictest157112-21kworker/2:022:06:292
151661995040,6cyclictest157112-21kworker/2:023:32:102
151661995027,7cyclictest126577-21kworker/2:220:00:022
15166099507,6cyclictest23-21ksoftirqd/123:52:471
15166099505,9cyclictest23-21ksoftirqd/122:20:001
15166099501,9cyclictest154241-21munin-run20:00:001
151659995024,11cyclictest12-21ksoftirqd/022:25:030
151659995021,12cyclictest217901-21ssh23:15:070
151661994941,5cyclictest157112-21kworker/2:000:26:442
151661994941,5cyclictest157112-21kworker/2:000:26:432
151661994941,5cyclictest126577-21kworker/2:220:30:002
151661994940,5cyclictest157112-21kworker/2:022:36:262
151661994940,5cyclictest157112-21kworker/2:022:36:262
151661994940,5cyclictest126577-21kworker/2:220:35:022
151660994917,9cyclictest131rcu_preempt19:25:011
15165999499,32cyclictest16850irq/134-800000021:18:130
151659994927,7cyclictest257845-21ssh00:39:240
151659994927,7cyclictest257845-21ssh00:39:240
15165999491,5cyclictest131rcu_preempt19:20:000
151659994912,33cyclictest16850irq/134-800000000:20:070
151664994829,15cyclictest275-21dbus-daemon23:20:013
151661994840,5cyclictest126577-21kworker/2:219:25:052
15166099487,6cyclictest23-21ksoftirqd/119:50:011
151659994829,14cyclictest275-21dbus-daemon20:20:010
151659994820,12cyclictest209938-21ssh22:58:510
151659994811,33cyclictest16850irq/134-800000023:00:130
151659994810,30cyclictest16850irq/134-800000023:47:150
151664994731,12cyclictest3467602-21systemd-journal20:15:473
151661994739,5cyclictest157112-21kworker/2:021:15:442
151661994739,5cyclictest126577-21kworker/2:221:00:012
151661994737,6cyclictest126577-21kworker/2:220:55:022
15166199472,10cyclictest154743-21sshd20:08:342
15166099479,7cyclictest23-21ksoftirqd/119:45:031
15166099477,9cyclictest23-21ksoftirqd/120:25:011
15166099476,10cyclictest23-21ksoftirqd/120:15:031
15166099476,10cyclictest23-21ksoftirqd/120:15:031
15166499461,42cyclictest1-21systemd22:30:003
151661994638,5cyclictest126577-21kworker/2:221:05:012
151661994638,5cyclictest126577-21kworker/2:221:05:012
151661994638,5cyclictest126577-21kworker/2:220:21:342
151661994638,5cyclictest126577-21kworker/2:220:00:442
151661994637,6cyclictest126577-21kworker/2:219:35:022
151661994628,7cyclictest157112-21kworker/2:022:49:192
151661994619,23cyclictest30-21ksoftirqd/219:40:002
15166099462,10cyclictest23-21ksoftirqd/119:15:001
151659994638,5cyclictest16950irq/224-800000020:40:000
151659994631,11cyclictest12-21ksoftirqd/019:54:590
151664994530,12cyclictest37-21ksoftirqd/300:10:013
151661994537,5cyclictest126577-21kworker/2:220:17:462
151661994521,5cyclictest30-21ksoftirqd/220:50:002
15166099456,8cyclictest23-21ksoftirqd/119:10:011
15166099455,8cyclictest23-21ksoftirqd/119:55:021
15166099453,9cyclictest23-21ksoftirqd/121:05:031
15166099453,9cyclictest23-21ksoftirqd/121:05:031
151660994532,9cyclictest23-21ksoftirqd/120:55:011
151659994529,12cyclictest154733-21sshd20:05:450
15165999452,3cyclictest131rcu_preempt19:40:000
151664994434,6cyclictest3467562-21sshd23:55:443
151664994429,12cyclictest37-21ksoftirqd/322:15:023
151661994437,4cyclictest126577-21kworker/2:220:41:062
151661994435,6cyclictest126577-21kworker/2:220:35:012
151661994430,10cyclictest30-21ksoftirqd/219:50:022
15166099445,10cyclictest364-21snmpd20:46:031
15166099443,10cyclictest23-21ksoftirqd/120:45:001
15166099441,11cyclictest23-21ksoftirqd/119:35:011
15166499433,4cyclictest131rcu_preempt22:19:163
151664994327,13cyclictest275-21dbus-daemon22:50:013
151664994327,10cyclictest37-21ksoftirqd/321:39:593
15166499432,3cyclictest131rcu_preempt19:35:013
15166099438,7cyclictest23-21ksoftirqd/120:30:021
15166099431,9cyclictest23-21ksoftirqd/120:10:011
15165999432,3cyclictest131rcu_preempt20:05:010
15165999431,38cyclictest157648-21munin-run21:05:020
15165999431,38cyclictest157648-21munin-run21:05:010
151664994232,6cyclictest153474-21munin-plugin-st19:45:013
151664994232,6cyclictest153474-21munin-plugin-st19:45:003
151661994233,6cyclictest126577-21kworker/2:221:10:012
15166099425,7cyclictest23-21ksoftirqd/120:35:011
151659994224,14cyclictest12-21ksoftirqd/020:29:590
151664994134,5cyclictest37-21ksoftirqd/322:56:313
151664994132,5cyclictest201081-21sh22:40:023
151664994132,5cyclictest201081-21sh22:40:023
151664994130,8cyclictest154996-21sshd20:13:353
151664994130,8cyclictest154996-21sshd20:13:353
151664994129,8cyclictest222440-21cron23:25:003
151664994128,10cyclictest275-21dbus-daemon20:35:023
151661994133,5cyclictest126577-21kworker/2:219:45:022
151661994133,5cyclictest126577-21kworker/2:219:45:012
151661994120,5cyclictest30-21ksoftirqd/219:30:012
151661994120,5cyclictest30-21ksoftirqd/219:30:002
151659994127,10cyclictest157341-21cron21:00:000
15165999411,3cyclictest131rcu_preempt20:00:020
151664994029,9cyclictest37-21ksoftirqd/322:30:173
151664994028,10cyclictest37-21ksoftirqd/321:29:593
151664994026,11cyclictest37-21ksoftirqd/321:31:063
151664994026,11cyclictest37-21ksoftirqd/321:31:063
151664994025,10cyclictest0-21swapper/319:30:013
151664994025,10cyclictest0-21swapper/319:30:013
151664994024,13cyclictest229519-21cron23:40:003
151664994024,13cyclictest229519-21cron23:40:003
15166199408,5cyclictest131rcu_preempt20:15:022
15166199408,5cyclictest131rcu_preempt20:15:022
151659994025,10cyclictest3467718-21lldpd19:48:350
15165999401,3cyclictest131rcu_preempt20:10:350
15165999401,3cyclictest131rcu_preempt20:10:340
15166499399,5cyclictest131rcu_preempt00:14:103
151664993933,3cyclictest371ksoftirqd/321:59:593
151664993930,5cyclictest37-21ksoftirqd/300:30:233
151664993929,7cyclictest37-21ksoftirqd/322:08:433
151664993928,8cyclictest37-21ksoftirqd/320:55:013
151664993926,10cyclictest275-21dbus-daemon23:25:483
151664993923,11cyclictest37-21ksoftirqd/321:50:033
151664993914,8cyclictest37-21ksoftirqd/322:20:453
151664993912,5cyclictest131rcu_preempt00:27:103
151664993912,5cyclictest131rcu_preempt00:27:103
151661993926,8cyclictest30-21ksoftirqd/219:55:002
15166099398,7cyclictest0-21swapper/119:26:421
15166099398,7cyclictest0-21swapper/119:26:411
15166099393,9cyclictest23-21ksoftirqd/119:45:011
151659993925,11cyclictest275-21dbus-daemon19:45:010
151659993925,11cyclictest275-21dbus-daemon19:45:000
15165999392,3cyclictest131rcu_preempt19:15:040
15165999391,3cyclictest131rcu_preempt19:35:020
15166499384,3cyclictest131rcu_preempt00:01:513
151664993829,7cyclictest37-21ksoftirqd/320:20:053
151664993824,11cyclictest37-21ksoftirqd/323:46:013
151664993824,11cyclictest37-21ksoftirqd/323:13:463
15166499382,3cyclictest131rcu_preempt20:00:013
151664993823,12cyclictest162625-21systemctl21:20:003
151664993822,12cyclictest3467602-21systemd-journal23:35:023
151664993816,5cyclictest131rcu_preempt22:05:023
151664993816,5cyclictest131rcu_preempt22:05:023
15166499381,5cyclictest131rcu_preempt21:45:193
151664993812,5cyclictest131rcu_preempt23:04:353
151659993833,3cyclictest16950irq/224-800000019:25:020
151659993828,7cyclictest156846-21sh20:50:000
151664993728,5cyclictest246539-21sh00:15:173
151664993726,8cyclictest37-21ksoftirqd/321:22:563
151664993726,8cyclictest37-21ksoftirqd/321:22:563
151664993726,8cyclictest157987-21cat21:10:013
151664993725,8cyclictest154596-21sed20:05:013
151664993724,11cyclictest37-21ksoftirqd/322:52:473
151664993722,11cyclictest37-21ksoftirqd/320:40:003
15165999372,31cyclictest0-21swapper/019:10:010
151659993723,10cyclictest3467602-21systemd-journal20:21:320
15166499364,3cyclictest131rcu_preempt19:40:013
151664993628,5cyclictest174621-21cron21:45:003
151664993628,5cyclictest174621-21cron21:45:003
151664993625,8cyclictest37-21ksoftirqd/323:10:023
151664993625,7cyclictest152327-21cat19:20:023
151661993628,5cyclictest126577-21kworker/2:219:15:132
15166199361,4cyclictest131rcu_preempt19:15:042
151659993626,6cyclictest309-21in:imuxsock20:45:010
15165999362,4cyclictest131rcu_preempt19:30:010
15165999362,4cyclictest131rcu_preempt19:30:010
151659993623,10cyclictest12-21ksoftirqd/020:30:020
151664993528,4cyclictest234583-21sh23:50:073
151664993527,5cyclictest160034-21sh21:14:283
151664993522,10cyclictest153896-21sed19:50:023
151664993427,4cyclictest151410-21kworker/u8:019:55:003
151664993419,12cyclictest3467718-21lldpd00:20:453
15166499341,4cyclictest131rcu_preempt19:25:053
151664993324,7cyclictest37-21ksoftirqd/319:13:093
151664993317,12cyclictest275-21dbus-daemon21:05:013
151664993317,12cyclictest275-21dbus-daemon21:05:013
15166499322,3cyclictest131rcu_preempt20:30:003
151664993221,7cyclictest309-21in:imuxsock20:50:013
151664993220,10cyclictest37-21ksoftirqd/320:45:003
151661993220,9cyclictest151732-21cut19:10:012
151664992814,10cyclictest151795-21sed19:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional