You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 19:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Fri Jul 31, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
64445399937,3cyclictest131rcu_preempt09:20:023
64445399937,3cyclictest131rcu_preempt09:20:013
64444499851,66cyclictest710308-21ssh11:15:270
644444998429,38cyclictest686529-21ssh10:25:300
644444998427,53cyclictest669001-21ssh09:48:420
644444998421,32cyclictest692777-21ssh10:39:020
644453998230,20cyclictest715376-21sh11:26:173
644453998128,14cyclictest738473-21ssh12:15:113
64444499812,74cyclictest733656-21latency_hist12:05:010
644444998027,49cyclictest696684-21ssh10:47:000
644444997921,53cyclictest16850irq/134-800000011:40:460
644453997825,15cyclictest744402-21sh12:27:513
644444997829,33cyclictest736812-21ssh12:11:440
644453997729,14cyclictest364-21snmpd09:42:513
644453997725,14cyclictest695547-21diskmemload10:44:563
64444499771,70cyclictest653093-21latency_hist09:15:010
644453997630,14cyclictest749516-21sh12:38:493
644453997626,16cyclictest691685-21sh10:36:223
644444997627,44cyclictest670473-21ssh09:51:400
644444997529,29cyclictest716821-21ssh11:29:380
64444499751,5cyclictest745297-21modprobe12:30:000
644444997431,26cyclictest720160-21ssh11:36:260
644444997431,26cyclictest720160-21ssh11:36:260
644453997328,15cyclictest0-21swapper/309:52:183
644453997328,14cyclictest680010-21sh10:12:083
644444997329,41cyclictest654824-21ssh09:18:580
644444997329,41cyclictest654824-21ssh09:18:580
64444499732,65cyclictest676730-21sort10:05:020
644444997320,44cyclictest16850irq/134-800000011:11:370
644444997320,44cyclictest16850irq/134-800000011:11:370
64444499722,29cyclictest683640-21munin-run10:20:010
644453997122,13cyclictest0-21swapper/311:43:223
644444997131,36cyclictest16950irq/224-800000010:11:060
644444997128,26cyclictest725962-21ssh11:48:520
644444997128,26cyclictest725962-21ssh11:48:520
64444499711,3cyclictest0-21swapper/011:54:100
64444499711,17cyclictest667246-21sed09:45:020
644451997012,23cyclictest691250-21ssh10:35:282
644444997023,43cyclictest748854-21ssh12:37:150
644444997021,44cyclictest16850irq/134-800000010:23:400
644444997020,45cyclictest745779-21ssh12:30:330
64444499701,53cyclictest714092-21ssh11:23:460
64444499692,4cyclictest676852-21sshd10:05:140
64444499692,4cyclictest676852-21sshd10:05:130
644444996921,43cyclictest16850irq/134-800000009:33:420
644444996919,45cyclictest16850irq/134-800000010:54:400
644444996919,45cyclictest16850irq/134-800000010:54:400
644453996820,14cyclictest644636-21munin-run07:15:013
644453996820,14cyclictest644636-21munin-run07:15:003
64444499683,60cyclictest657708-21ssh09:25:000
64444499682,61cyclictest650642-21ssh09:10:010
64444499682,3cyclictest0-21swapper/012:10:020
644444996729,34cyclictest703975-21ssh11:02:200
64444499672,17cyclictest645596-21date07:30:030
644444996721,41cyclictest16850irq/134-800000011:08:230
644444996715,26cyclictest16850irq/134-800000009:27:400
644444996715,26cyclictest16850irq/134-800000009:27:400
644453996522,11cyclictest0-21swapper/311:59:483
644453996522,11cyclictest0-21swapper/311:59:483
644451996553,8cyclictest629740-21kworker/2:207:55:032
64444899651,5cyclictest0-21swapper/110:15:021
64444499652,3cyclictest0-21swapper/009:56:140
644453996419,13cyclictest650613-21diskmemload10:04:563
644448996447,14cyclictest0-21swapper/110:25:021
64444499642,58cyclictest645911-21sshd07:39:130
644444996423,36cyclictest739994-21rm12:18:400
644444996423,25cyclictest664056-21ssh09:38:180
64444499641,7cyclictest648305-21latency_hist08:25:010
644453996332,27cyclictest3467602-21systemd-journal11:25:013
644453996319,14cyclictest650613-21diskmemload10:46:103
64445399621,3cyclictest131rcu_preempt12:35:013
64444499621,3cyclictest71918450rm11:34:430
644444996212,46cyclictest741769-21sshd12:22:140
64444499619,33cyclictest16850irq/134-800000010:33:560
644444996123,34cyclictest702289-21sh10:59:060
64444499611,2cyclictest0-21swapper/011:59:380
64444499611,2cyclictest0-21swapper/011:59:380
644453996036,17cyclictest668026-21ssh09:46:323
644453996024,12cyclictest709200-21diskmemload11:13:303
644453996024,12cyclictest709200-21diskmemload11:13:303
644453996016,14cyclictest650613-21diskmemload11:46:353
644453996016,14cyclictest650613-21diskmemload11:46:353
644451996015,13cyclictest722282-21ssh11:40:522
644444996032,26cyclictest150-21pr/ttyS208:19:590
64444499602,6cyclictest695619-21munin-run10:45:010
644453995946,10cyclictest678744-21ssh10:09:423
644453995946,10cyclictest678744-21ssh10:09:423
644453995945,10cyclictest698974-21ssh10:51:463
644453995945,10cyclictest698974-21ssh10:51:463
644453995920,12cyclictest0-21swapper/312:06:573
644453995919,12cyclictest650613-21diskmemload11:54:513
64444499591,7cyclictest647246-21munin-run08:05:010
644453995817,12cyclictest650613-21diskmemload09:38:453
644448995838,8cyclictest705196-21ssh11:05:021
644444995824,3cyclictest0-21swapper/008:40:140
644453995744,10cyclictest684344-21ssh10:21:083
644453995744,10cyclictest661984-21ssh09:34:073
64444499572,51cyclictest646462-21munin-run07:50:010
64444499571,15cyclictest650355-21latency_hist09:05:000
644444995618,33cyclictest309-21in:imuxsock08:59:590
644453995541,10cyclictest707116-21ssh11:09:073
644453995540,12cyclictest682987-21ssh10:18:333
644453995539,13cyclictest656344-21ssh09:21:573
644453995532,19cyclictest733191-21ssh12:04:173
644453995531,19cyclictest719466-21wc11:35:023
644453995514,11cyclictest650613-21diskmemload09:58:523
64444499552,48cyclictest646715-21munin-run07:55:000
64444499552,42cyclictest644450-21latency_hist07:10:020
64444499551,9cyclictest48-21kcompactd007:30:160
644453995436,14cyclictest650624-21ssh09:10:003
644453995432,18cyclictest720354-21ssh11:37:023
644453995432,18cyclictest720354-21ssh11:37:023
644453995429,20cyclictest710556-21sh11:16:083
644453995413,12cyclictest650613-21diskmemload12:22:003
644453995413,10cyclictest650613-21diskmemload10:25:103
64444499542,47cyclictest364-21snmpd08:31:310
64444499541,49cyclictest0-21swapper/007:13:140
64444499541,49cyclictest0-21swapper/007:13:140
644444995411,36cyclictest0-21swapper/008:00:000
644453995338,12cyclictest658233-21ssh09:25:433
644453995338,12cyclictest658233-21ssh09:25:433
644453995337,12cyclictest737687-21ssh12:13:503
644453995332,16cyclictest48-21kcompactd011:00:263
644453995328,21cyclictest652585-21sh09:14:203
644453995318,5cyclictest648620-21latency_hist08:30:023
64444899534,35cyclictest674281-21rm10:00:021
644448995337,12cyclictest1-21systemd10:35:011
644448995336,12cyclictest275-21dbus-daemon09:55:011
64444499533,44cyclictest311-21rs:main0
64444499531,6cyclictest648702-21latency_hist08:30:020
644453995239,10cyclictest689265-21ssh10:31:243
644451995232,15cyclictest3467602-21systemd-journal12:40:002
644448995234,12cyclictest275-21dbus-daemon10:40:011
644448995233,14cyclictest275-21dbus-daemon10:55:011
644448995233,14cyclictest275-21dbus-daemon10:55:011
64444499522,45cyclictest0-21swapper/007:18:520
64444499521,4cyclictest647509-21munin-run08:10:020
644453995135,13cyclictest701420-21ssh10:57:053
644453995132,15cyclictest647711-21sshd08:10:153
644453995128,16cyclictest647257-21systemd-run08:05:013
644453995125,20cyclictest650247-21ls09:00:023
644451995143,5cyclictest657702-21kworker/2:209:40:022
644451995138,10cyclictest699884-21ssh10:53:592
644451995138,10cyclictest699884-21ssh10:53:592
644451995135,12cyclictest708961-21ssh11:12:592
644451995135,12cyclictest708961-21ssh11:12:592
64444499513,41cyclictest311-21rs:main0
64444499512,45cyclictest364-21snmpd07:21:220
64444499512,44cyclictest48-21kcompactd008:10:130
644453995030,12cyclictest649313-21modprobe08:45:003
644453995029,16cyclictest648174-21latency_hist08:20:023
644453995026,19cyclictest648896-21latency_hist08:35:003
644451995037,10cyclictest715182-21ssh11:25:532
644451995031,15cyclictest0-21swapper/212:05:012
644451995021,24cyclictest690951-21cut10:35:022
644448995029,14cyclictest3467602-21systemd-journal07:59:591
644448995012,21cyclictest645644-21kworker/u8:308:15:021
64444499501,45cyclictest646186-21sshd07:43:150
644453994934,11cyclictest650428-21sed09:05:003
644453994927,17cyclictest645170-21latency_hist07:25:023
644453994926,19cyclictest644867-21sshd07:17:153
644453994925,19cyclictest645963-21true07:40:013
644451994936,9cyclictest719659-21ssh11:35:232
644451994936,9cyclictest719659-21ssh11:35:232
644451994934,12cyclictest684673-21ssh10:21:492
644451994934,12cyclictest667419-21ssh09:45:082
644451994928,17cyclictest652249-21sh09:13:282
644451994923,22cyclictest676797-21sh10:05:072
644451994923,22cyclictest676797-21sh10:05:072
64444499493,41cyclictest284-21systemd-logind08:55:010
644453994834,9cyclictest1-21systemd07:50:013
644453994827,15cyclictest1-21systemd08:40:013
644453994822,21cyclictest1-21systemd07:58:143
644453994817,26cyclictest647684-21cat08:10:023
644451994840,5cyclictest674153-21kworker/2:011:33:352
644451994835,10cyclictest745924-21ssh12:30:512
644451994835,10cyclictest726164-21ssh11:49:272
644451994835,10cyclictest726164-21ssh11:49:272
644451994835,10cyclictest670469-21ssh09:51:402
644451994833,12cyclictest714242-21ssh11:24:122
644451994826,18cyclictest650613-21diskmemload11:55:482
644451994826,18cyclictest650613-21diskmemload11:55:482
644448994835,9cyclictest1-21systemd12:05:001
644448994834,11cyclictest3467602-21systemd-journal09:20:141
644448994833,11cyclictest23-21ksoftirqd/108:55:021
644448994830,11cyclictest688468-21grep10:30:011
644453994732,11cyclictest275-21dbus-daemon07:45:013
644453994731,11cyclictest649587-21unin-run08:49:593
644451994738,6cyclictest657702-21kworker/2:212:10:362
644451994735,10cyclictest653243-21ssh09:15:252
644451994735,10cyclictest653243-21ssh09:15:252
644451994734,9cyclictest646952-21cron08:00:002
644451994732,12cyclictest693666-21ssh10:40:302
644451994730,12cyclictest275-21dbus-daemon08:20:012
644451994723,20cyclictest702745-21sendmail11:00:002
644451994722,21cyclictest649574-21/usr/sbin/munin08:50:012
644451994716,5cyclictest30-21ksoftirqd/209:58:552
644451994714,5cyclictest30-21ksoftirqd/211:51:272
64445399465,12cyclictest646686-21sshd07:52:143
644453994626,12cyclictest645383-21latency_hist07:25:023
644451994633,10cyclictest739846-21ssh12:18:192
644451994633,10cyclictest364-21snmpd08:23:302
644451994623,17cyclictest674645-21ssh10:00:372
644451994622,20cyclictest705785-21ssh11:05:562
644451994621,20cyclictest649276-21cat08:40:022
644448994633,10cyclictest646676-21sshd07:50:141
644448994627,15cyclictest1-21systemd08:50:001
644448994626,15cyclictest275-21dbus-daemon12:15:001
644448994625,15cyclictest3467602-21systemd-journal07:44:591
644448994624,16cyclictest3467602-21systemd-journal08:30:011
644453994527,15cyclictest648010-21latency_hist08:20:023
644451994533,9cyclictest697252-21ssh10:48:212
644451994533,8cyclictest3467602-21systemd-journal07:16:152
644451994529,13cyclictest681624-21ssh10:15:202
644451994529,13cyclictest1-21systemd08:30:002
644451994526,15cyclictest743242-21ssh12:25:142
644451994526,14cyclictest680831-21ssh10:14:022
644451994523,18cyclictest711027-21sh11:17:072
644451994522,20cyclictest735042-21rm12:08:042
64444899454,3cyclictest131rcu_preempt09:15:341
64444899454,3cyclictest131rcu_preempt09:15:341
644448994535,6cyclictest735856-21cron12:10:001
644448994529,12cyclictest23-21ksoftirqd/111:10:021
644448994528,13cyclictest275-21dbus-daemon10:10:011
644448994528,13cyclictest275-21dbus-daemon10:10:011
644448994527,14cyclictest275-21dbus-daemon09:50:011
644448994526,15cyclictest275-21dbus-daemon11:00:001
644453994432,9cyclictest364-21snmpd08:51:473
644453994431,9cyclictest645651-21sshd07:32:143
64445199446,5cyclictest131rcu_preempt08:45:002
644451994432,10cyclictest657153-21ssh09:23:482
644451994431,11cyclictest687476-21ssh10:27:512
644451994423,17cyclictest665271-21sh09:40:392
644451994423,17cyclictest645664-21cron07:34:592
644451994422,18cyclictest660849-21ssh09:31:242
644451994421,18cyclictest743143-21latency_hist12:25:022
644451994421,18cyclictest645970-21unin-run07:40:022
644448994430,11cyclictest120-21kswapd011:15:011
644448994430,11cyclictest120-21kswapd011:15:011
644448994429,9cyclictest309-21in:imuxsock09:30:031
64444899442,23cyclictest221rcuc/107:40:031
644448994420,5cyclictest23-21ksoftirqd/110:49:301
644453994325,12cyclictest644478-21latency_hist07:10:023
644451994325,15cyclictest704897-21sh11:04:262
644451994323,16cyclictest658303-21ssh09:25:512
644451994323,16cyclictest658303-21ssh09:25:512
644451994323,16cyclictest647721-21sshd08:12:152
644451994319,19cyclictest650372-21munin-run09:05:012
644448994333,6cyclictest309-21in:imuxsock12:35:011
64444899432,5cyclictest131rcu_preempt10:45:001
644451994229,9cyclictest1-21systemd07:45:012
644451994221,16cyclictest647451-21latency_hist08:05:022
644448994228,11cyclictest23-21ksoftirqd/110:20:011
644448994227,4cyclictest23-21ksoftirqd/111:29:371
644448994227,10cyclictest275-21dbus-daemon07:20:001
644451994127,11cyclictest30-21ksoftirqd/208:07:052
644451994119,18cyclictest646482-21cat07:50:012
644451994119,17cyclictest645465-21latency_hist07:30:022
644451994119,17cyclictest644551-21latency_hist07:10:022
644451994115,21cyclictest650780-21ssh09:10:022
644448994130,7cyclictest645169-21sh07:25:001
644448994127,10cyclictest23-21ksoftirqd/112:20:011
644448994125,12cyclictest647996-21cron08:20:001
64445199402,7cyclictest650087-21munin-run09:00:002
644451994027,10cyclictest1-21systemd08:35:002
644451993921,13cyclictest645147-21sshd07:22:132
644448993927,9cyclictest23-21ksoftirqd/111:58:491
644448993927,9cyclictest23-21ksoftirqd/111:58:491
644448993925,10cyclictest0-21swapper/107:30:021
644448993923,11cyclictest23-21ksoftirqd/109:04:481
644448993918,13cyclictest310-21in:imklog11:51:491
644451993826,9cyclictest48-21kcompactd007:10:182
644451993826,9cyclictest48-21kcompactd007:10:172
644451993825,10cyclictest30-21ksoftirqd/208:54:362
64444899383,3cyclictest131rcu_preempt11:37:041
64444899383,3cyclictest131rcu_preempt11:37:041
644448993828,7cyclictest23-21ksoftirqd/111:44:591
644448993827,9cyclictest660150-21latency_hist09:30:021
644448993827,9cyclictest660150-21latency_hist09:30:011
644448993827,9cyclictest651293-21sshd09:11:151
644448993826,9cyclictest23-21ksoftirqd/108:40:011
64444899381,3cyclictest131rcu_preempt12:38:301
644448993813,21cyclictest648301-21unin-run08:25:001
64444899376,7cyclictest705139-21kworker/u8:311:45:351
64444899376,7cyclictest705139-21kworker/u8:311:45:351
64444899373,6cyclictest676668-21ssh10:05:021
644448993726,6cyclictest309-21in:imuxsock07:35:001
644448993720,13cyclictest3467602-21systemd-journal09:00:011
644448993719,15cyclictest23-21ksoftirqd/109:45:021
644448993718,14cyclictest275-21dbus-daemon09:10:011
644448993718,13cyclictest0-21swapper/112:25:481
64444899364,3cyclictest131rcu_preempt11:24:321
64444899363,6cyclictest131rcu_preempt09:40:001
644448993616,15cyclictest3467602-21systemd-journal07:15:011
644448993616,15cyclictest3467602-21systemd-journal07:15:011
644448993615,5cyclictest23-21ksoftirqd/112:22:401
64444899355,6cyclictest131rcu_preempt11:17:301
644448993422,8cyclictest3467602-21systemd-journal07:47:151
644448993420,11cyclictest23-21ksoftirqd/108:05:471
644448993325,5cyclictest718848-21sh11:34:021
644448993319,9cyclictest275-21dbus-daemon08:45:011
64444899314,4cyclictest131rcu_preempt08:31:141
644448993122,7cyclictest23-21ksoftirqd/108:02:031
64444899266,14cyclictest0-21swapper/107:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional