You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 14:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Mon Aug 31, 2026 00:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4398279910231,36cyclictest450335-21sh21:19:271
43982199942,12cyclictest495174-21sort22:55:010
43982199913,14cyclictest450642-21modprobe21:20:000
439821998733,50cyclictest520996-21ssh23:50:020
439821998523,57cyclictest12-21ksoftirqd/023:35:030
439821998523,57cyclictest12-21ksoftirqd/023:35:020
43982199852,14cyclictest444434-21munin-run20:40:010
43982199852,14cyclictest444434-21munin-run20:40:000
439821998418,56cyclictest16850irq/134-800000021:20:440
43982199832,64cyclictest447310-21ssh21:12:470
43982199832,64cyclictest447310-21ssh21:12:460
439821998213,63cyclictest0-21swapper/000:40:000
439821998115,7cyclictest16850irq/134-800000000:04:150
43982199801,75cyclictest0-21swapper/022:46:540
43982199801,75cyclictest0-21swapper/022:46:530
43982199794,18cyclictest462267-21modprobe21:44:590
439821997729,30cyclictest523430-21ssh23:55:100
439821997717,5cyclictest16850irq/134-800000000:22:470
439821997717,43cyclictest16850irq/134-800000023:24:100
439821997717,43cyclictest16850irq/134-800000023:24:090
43982199771,60cyclictest474805-21ssh22:11:120
439821997627,32cyclictest496935-21ssh22:59:030
439821997538,20cyclictest469158-21ssh21:59:400
439821997538,20cyclictest469158-21ssh21:59:400
439821997326,43cyclictest466381-21ssh21:53:340
439821997322,33cyclictest457539-21ssh21:34:560
439821997318,50cyclictest16850irq/134-800000023:11:150
439821997317,7cyclictest16850irq/134-800000023:46:410
43982199731,2cyclictest0-21swapper/022:30:230
439821997312,43cyclictest16850irq/134-800000023:29:550
439821997228,27cyclictest477037-21ssh22:16:030
439821997228,27cyclictest477037-21ssh22:16:020
439821997225,30cyclictest486356-21ssh22:35:540
439821997224,41cyclictest309-21in:imuxsock23:03:150
43982799712,39cyclictest521936-21ssh23:52:101
43982199716,24cyclictest441310-21cron19:40:000
439821997118,35cyclictest16850irq/134-800000000:14:140
43982199711,66cyclictest529343-21ssh00:08:010
43982199696,14cyclictest446005-21modprobe21:10:000
43982199693,9cyclictest481125-21latency_hist22:25:010
43982199692,12cyclictest455435-21latency_hist21:30:020
439821996917,7cyclictest459220-21ssh21:38:270
43982199691,51cyclictest0-21swapper/022:06:590
43982799682,32cyclictest457533-21ssh21:34:501
439821996828,24cyclictest463044-21ssh21:46:110
439827996729,34cyclictest516895-21sh23:41:121
439827996727,36cyclictest476356-21pluginstate22:15:011
43982799672,31cyclictest445995-21modprobe21:10:011
439821996740,24cyclictest16950irq/224-800000020:25:010
439821996726,37cyclictest0-21swapper/000:26:520
439827996628,33cyclictest502323-21sh23:10:081
43982199669,16cyclictest489813-21ssh22:43:280
43982199662,59cyclictest0-21swapper/019:20:000
43982799652,59cyclictest470021-21ssh22:01:051
43982799651,59cyclictest494040-21ssh22:52:311
43982799651,38cyclictest512305-21ssh23:31:301
43982799651,38cyclictest512305-21ssh23:31:291
439821996530,31cyclictest1-21systemd20:15:010
439821996530,30cyclictest0-21swapper/023:20:000
43982199651,60cyclictest0-21swapper/000:15:160
43982199651,60cyclictest0-21swapper/000:15:160
439831996417,8cyclictest30-21ksoftirqd/221:40:012
43982799642,58cyclictest526951-21ssh00:02:541
43982799642,58cyclictest461888-21ssh21:44:041
43982799642,57cyclictest533755-21ssh00:17:241
43982799642,57cyclictest533755-21ssh00:17:231
439827996420,39cyclictest3467602-21systemd-journal22:35:021
43982799641,59cyclictest530670-21ssh00:10:431
43982799641,59cyclictest525169-21ssh23:59:221
43982799641,59cyclictest519082-21ssh23:45:511
43982799641,59cyclictest496596-21ssh22:58:081
43982799641,59cyclictest490441-21ssh22:44:481
43982799641,59cyclictest481442-21ssh22:25:251
43982799641,59cyclictest472227-21ssh22:05:461
43982799641,59cyclictest468248-21ssh21:57:321
43982799641,59cyclictest468248-21ssh21:57:321
43982799641,59cyclictest465204-21ssh21:50:471
43982799641,59cyclictest463326-21ssh21:46:571
43982799641,59cyclictest458115-21ssh21:35:461
43982799641,58cyclictest540216-21ssh00:31:161
439821996427,22cyclictest516562-21ssh23:40:310
43982799632,56cyclictest446267-21ssh21:10:071
43982799632,56cyclictest446267-21ssh21:10:071
43982799631,58cyclictest542635-21ssh00:36:281
43982799631,58cyclictest539292-21ssh00:29:401
43982799631,58cyclictest514976-21ssh23:37:211
43982799631,58cyclictest510272-21ssh23:27:051
43982799631,58cyclictest505376-21ssh23:16:471
43982799631,58cyclictest499991-21ssh23:05:041
43982799631,58cyclictest497964-21ssh23:00:511
43982799631,58cyclictest491558-21ssh22:47:071
43982799631,58cyclictest491558-21ssh22:47:061
43982799631,58cyclictest486225-21ssh22:35:371
43982799631,58cyclictest477945-21ssh22:18:191
43982799631,58cyclictest477945-21ssh22:18:181
43982799631,58cyclictest453910-21ssh21:26:431
43982799631,58cyclictest450890-21ssh21:20:051
43982199634,15cyclictest443388-21modprobe20:20:000
43982199633,21cyclictest440560-21grep19:25:000
439821996315,9cyclictest445994-21diskmemload22:00:300
43982799622,56cyclictest507585-21ssh23:21:231
43982799622,56cyclictest507585-21ssh23:21:221
43982799622,56cyclictest48-21kcompactd000:20:201
43982799621,58cyclictest528820-21ssh00:06:441
43982799621,57cyclictest479385-21ssh22:20:581
43982799621,55cyclictest48-21kcompactd019:48:471
439821996218,41cyclictest541811-21rm00:35:010
439835996117,13cyclictest521039-21sh23:50:073
43982799611,56cyclictest444676-21sshd20:41:151
43982799611,55cyclictest440262-21sshd19:19:161
43982199614,9cyclictest1-21systemd19:45:010
439821996115,3cyclictest487459-21kworker/u8:023:07:180
439835996016,14cyclictest440705-21sed19:25:023
439831996015,7cyclictest30-21ksoftirqd/221:18:292
43982799601,55cyclictest441283-21sshd19:36:211
43982199603,17cyclictest1-21systemd20:50:010
43982199603,13cyclictest516100-21cron23:39:590
439835995916,14cyclictest508306-21sh23:23:073
439835995916,14cyclictest508306-21sh23:23:063
439831995919,10cyclictest30-21ksoftirqd/223:40:002
43982799592,52cyclictest364-21snmpd19:51:271
43982799591,54cyclictest442322-21sshd19:57:151
43982199594,10cyclictest284-21systemd-logind19:30:000
43982199593,19cyclictest445488-21sendmail-msp21:00:000
439821995924,31cyclictest481816-21ssh22:26:210
439821995920,33cyclictest444335-21latency_hist20:35:020
43983599589,10cyclictest0-21swapper/321:15:023
43983599589,10cyclictest0-21swapper/321:15:013
43983199585,11cyclictest30-21ksoftirqd/223:23:012
43983199585,11cyclictest30-21ksoftirqd/223:23:012
43982799581,53cyclictest445723-21sshd21:04:151
43982799581,53cyclictest444929-21sshd20:46:331
43982799581,53cyclictest441074-21munin-run19:35:021
43982799581,52cyclictest364-21snmpd20:31:361
43982799581,52cyclictest364-21snmpd19:21:211
43982199583,17cyclictest441053-21modprobe19:35:000
439835995720,12cyclictest524738-21ssh23:58:153
439835995713,10cyclictest131rcu_preempt19:35:023
439831995745,6cyclictest202273-21kworker/2:020:00:002
439831995715,9cyclictest30-21ksoftirqd/221:24:392
439831995713,8cyclictest30-21ksoftirqd/223:13:122
43982799573,50cyclictest443684-21/usr/sbin/munin20:25:011
439827995718,35cyclictest443996-21cat20:30:031
43982799571,52cyclictest48-21kcompactd019:40:281
43982199575,12cyclictest284-21systemd-logind20:30:010
43982199572,16cyclictest442626-21munin-run20:05:000
439821995715,4cyclictest0-21swapper/019:11:200
439831995612,8cyclictest30-21ksoftirqd/222:13:042
43982799561,51cyclictest364-21snmpd19:26:221
43982799561,50cyclictest48-21kcompactd020:05:261
43982199562,51cyclictest439853-21latency_hist19:10:020
43982199562,51cyclictest439853-21latency_hist19:10:020
43982199561,50cyclictest0-21swapper/019:58:150
43982199561,2cyclictest0-21swapper/020:10:030
439831995534,16cyclictest3467602-21systemd-journal19:25:012
43982799552,48cyclictest364-21snmpd19:10:161
43982799551,50cyclictest48-21kcompactd020:37:371
43982799551,50cyclictest48-21kcompactd020:37:361
43982799551,50cyclictest442596-21sshd20:01:151
43982799551,50cyclictest3467562-21sshd20:52:161
439835995417,12cyclictest510719-21ssh23:28:133
439831995439,12cyclictest3467602-21systemd-journal20:10:152
439831995419,7cyclictest30-21ksoftirqd/221:45:592
439831995410,7cyclictest30-21ksoftirqd/223:27:482
43982799541,49cyclictest364-21snmpd20:11:311
43982199546,16cyclictest445761-21cat21:05:020
43983599534,19cyclictest0-21swapper/323:14:203
439831995318,11cyclictest30-21ksoftirqd/221:44:132
43982799532,47cyclictest445503-21munin-run21:00:011
43982199534,9cyclictest1-21systemd20:45:000
43982199532,11cyclictest442096-21munin-plugin-st19:55:010
439821995316,33cyclictest441864-21cat19:50:020
439835995234,14cyclictest443651-21sshd20:23:153
439835995229,18cyclictest445708-21sendmail-mta21:01:043
43983599522,15cyclictest535583-21ssh00:21:223
43983599522,13cyclictest468066-21ssh21:57:003
43983599522,13cyclictest468066-21ssh21:56:593
439835995215,11cyclictest457261-21ssh21:34:143
439831995243,6cyclictest441863-21kworker/2:221:35:012
439831995214,12cyclictest30-21ksoftirqd/223:43:122
439831995213,8cyclictest30-21ksoftirqd/222:27:002
439821995214,29cyclictest0-21swapper/020:55:020
439835995127,18cyclictest443909-21cron20:30:013
439835995112,6cyclictest131rcu_preempt22:50:013
439835995112,6cyclictest131rcu_preempt22:50:013
43983199517,11cyclictest30-21ksoftirqd/223:07:142
43982799511,46cyclictest364-21snmpd20:17:571
439835995034,13cyclictest0-21swapper/322:28:543
439835995028,17cyclictest488153-21cron22:40:003
439835995015,12cyclictest520015-21ssh23:48:133
43983199509,23cyclictest521033-21kworker/u8:300:15:012
439831995041,6cyclictest483443-21kworker/2:123:17:042
439831995034,13cyclictest30-21ksoftirqd/222:45:022
439831995012,10cyclictest30-21ksoftirqd/221:59:592
439831995012,10cyclictest30-21ksoftirqd/221:59:582
439831995011,8cyclictest30-21ksoftirqd/222:58:212
439835994934,11cyclictest516098-21cron23:39:593
439835994933,11cyclictest3467602-21systemd-journal23:19:593
439835994924,21cyclictest478856-21cut22:20:033
439835994924,21cyclictest478856-21cut22:20:023
439835994915,12cyclictest533407-21ssh00:16:423
439835994915,12cyclictest533407-21ssh00:16:423
439835994913,6cyclictest131rcu_preempt21:17:463
43983199496,6cyclictest30-21ksoftirqd/200:28:292
439831994940,6cyclictest30-21ksoftirqd/222:20:532
439831994913,10cyclictest0-21swapper/222:49:192
439831994913,10cyclictest0-21swapper/222:49:182
439835994832,11cyclictest3467602-21systemd-journal19:54:153
439835994829,15cyclictest275-21dbus-daemon00:30:013
439835994822,20cyclictest458877-21ssh21:37:323
439835994814,12cyclictest475121-21ssh22:12:053
43983199489,8cyclictest30-21ksoftirqd/222:19:552
43983199489,8cyclictest30-21ksoftirqd/222:19:552
43983199483,6cyclictest30-21ksoftirqd/200:20:182
43982799482,40cyclictest439865-21cat19:10:021
43982799482,40cyclictest439865-21cat19:10:021
439835994731,13cyclictest542036-21ssh00:35:033
439835994730,11cyclictest442334-21cron20:00:013
439835994726,18cyclictest497462-21latency_hist23:00:013
439835994726,16cyclictest452368-21ssh21:23:323
439835994723,19cyclictest462563-21sh21:45:103
439835994710,7cyclictest131rcu_preempt22:31:213
43983199476,7cyclictest30-21ksoftirqd/223:51:192
43983199476,7cyclictest30-21ksoftirqd/200:08:152
43983199475,7cyclictest30-21ksoftirqd/222:36:042
439831994712,7cyclictest30-21ksoftirqd/223:50:002
439831994710,5cyclictest131rcu_preempt23:56:272
439835994631,9cyclictest364-21snmpd21:05:523
439835994625,18cyclictest541991-21latency_hist00:35:023
439835994625,18cyclictest467013-21ssh21:55:013
439835994625,16cyclictest499290-21ssh23:04:043
439835994624,17cyclictest530894-21sh00:11:123
439835994624,17cyclictest501894-21ssh23:09:313
439835994622,20cyclictest473671-21sh22:09:103
439835994621,21cyclictest494742-21ssh22:54:123
43983199464,9cyclictest30-21ksoftirqd/222:05:562
439831994630,12cyclictest275-21dbus-daemon21:55:022
439831994611,11cyclictest30-21ksoftirqd/221:13:202
439831994611,11cyclictest30-21ksoftirqd/221:13:192
439831994610,10cyclictest30-21ksoftirqd/221:27:592
43983599453,4cyclictest131rcu_preempt22:42:183
439835994530,13cyclictest526191-21ssh00:01:083
439835994530,12cyclictest517258-21ssh23:42:083
439835994529,12cyclictest455253-21cron21:30:013
439835994528,13cyclictest479074-21ssh22:20:183
439835994527,14cyclictest528214-21ssh00:05:213
439835994523,18cyclictest444954-21latency_hist20:50:023
43983199456,8cyclictest30-21ksoftirqd/222:50:182
439831994515,18cyclictest0-21swapper/222:01:422
439831994513,7cyclictest0-21swapper/220:44:262
439831994512,8cyclictest0-21swapper/221:01:322
439835994430,12cyclictest470905-21ssh22:03:053
439835994429,12cyclictest440791-21sshd19:29:153
439835994428,12cyclictest511954-21ssh23:30:353
439835994428,12cyclictest511954-21ssh23:30:343
439835994425,16cyclictest461240-21ssh21:42:323
439835994424,16cyclictest441422-21latency_hist19:40:023
439835994423,17cyclictest442623-21latency_hist20:05:013
43983199448,4cyclictest131rcu_preempt00:37:192
43983199445,6cyclictest30-21ksoftirqd/223:04:122
439831994437,4cyclictest483443-21kworker/2:122:32:362
43983199443,10cyclictest30-21ksoftirqd/200:32:322
439831994414,6cyclictest0-21swapper/220:50:112
439831994411,9cyclictest0-21swapper/220:45:352
439835994328,11cyclictest3467602-21systemd-journal20:57:323
439835994323,16cyclictest3467602-21systemd-journal20:16:333
43983199438,9cyclictest0-21swapper/220:36:232
43983199438,9cyclictest0-21swapper/220:36:222
43983199435,8cyclictest30-21ksoftirqd/200:04:522
439831994326,13cyclictest443399-21systemctl20:20:002
439831994320,18cyclictest0-21swapper/219:55:012
439831994313,5cyclictest0-21swapper/220:57:242
439831994312,8cyclictest0-21swapper/219:40:292
439831994312,7cyclictest0-21swapper/219:27:382
439835994227,11cyclictest364-21snmpd20:41:383
439835994225,11cyclictest444176-21munin-plugin-st20:35:003
439835994224,14cyclictest444467-21systemd-run20:40:023
439835994224,14cyclictest444467-21systemd-run20:40:013
439831994226,12cyclictest30-21ksoftirqd/223:34:072
439831994226,12cyclictest30-21ksoftirqd/223:34:062
439831994214,5cyclictest0-21swapper/220:02:342
439831994212,6cyclictest0-21swapper/219:13:232
439831994211,7cyclictest0-21swapper/220:07:492
439831994211,7cyclictest0-21swapper/219:30:362
439835994129,9cyclictest439998-21sshd19:13:153
439835994118,19cyclictest445271-21grep20:55:023
43983199413,8cyclictest30-21ksoftirqd/200:16:372
43983199413,8cyclictest30-21ksoftirqd/200:16:362
439835994030,7cyclictest441563-21sshd19:41:223
439835994027,9cyclictest443100-21sshd20:11:163
439835994027,10cyclictest441825-21sshd19:49:153
43983199405,7cyclictest30-21ksoftirqd/219:40:022
439831994011,6cyclictest0-21swapper/220:33:012
439831994010,7cyclictest0-21swapper/221:05:502
439835993927,9cyclictest364-21snmpd19:19:533
439835993924,11cyclictest442892-21systemd-run20:10:013
43983199399,8cyclictest0-21swapper/219:19:362
439831993822,13cyclictest1-21systemd20:30:012
439831993710,5cyclictest0-21swapper/220:22:252
439835993618,14cyclictest439839-21latency_hist19:10:023
439835993618,14cyclictest439839-21latency_hist19:10:023
439831993520,11cyclictest439913-21sed19:10:022
439831993520,11cyclictest439913-21sed19:10:022
439831993325,6cyclictest30-21ksoftirqd/219:46:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional